-- Logs begin at Mon 2022-06-27 08:39:53 UTC, end at Mon 2022-06-27 09:57:13 UTC. -- Jun 27 09:13:55.969403 dv-cinder-847730-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 27 09:13:55.979147 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 27 09:13:55 2022] *** Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: nodename: dv-cinder-847730-1-iscsi Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: machine: x86_64 Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: clock source: unix Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: pcre jit disabled Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: detected number of CPU cores: 4 Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: current working directory: / Jun 27 09:13:55.980596 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: detected binary path: /usr/bin/uwsgi-core Jun 27 09:13:55.981119 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: your processes number limit is 47680 Jun 27 09:13:55.981119 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: your memory page size is 4096 bytes Jun 27 09:13:55.981119 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: detected max file descriptor number: 2048 Jun 27 09:13:55.981119 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: lock engine: pthread robust mutexes Jun 27 09:13:55.981119 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: thunder lock: enabled Jun 27 09:13:55.981119 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 27 09:13:55.981119 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 27 09:13:56.024739 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: Python main interpreter initialized at 0x55eec97939c0 Jun 27 09:13:56.024739 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: python threads support enabled Jun 27 09:13:56.024739 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: your server socket listen backlog is limited to 100 connections Jun 27 09:13:56.024739 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: your mercy for graceful operations on workers is 90 seconds Jun 27 09:13:56.025131 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: mapped 403077 bytes (393 KB) for 2 cores Jun 27 09:13:56.025209 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: *** Operational MODE: preforking *** Jun 27 09:13:56.025441 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: *** uWSGI is running in multiple interpreter mode *** Jun 27 09:13:56.025441 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: spawned uWSGI master process (pid: 80333) Jun 27 09:13:56.025685 dv-cinder-847730-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 27 09:13:56.027252 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: spawned uWSGI worker 1 (pid: 80344, cores: 1) Jun 27 09:13:56.027252 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: spawned uWSGI worker 2 (pid: 80345, cores: 1) Jun 27 09:13:56.027252 dv-cinder-847730-1-iscsi devstack@keystone.service[80333]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 27 09:14:01.537073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 09:14:01.537073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: from cryptography.utils import int_from_bytes Jun 27 09:14:01.537819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 09:14:01.537819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: from cryptography.utils import int_from_bytes Jun 27 09:14:01.538215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 09:14:01.538215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: from cryptography.utils import int_from_bytes Jun 27 09:14:01.538320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 09:14:01.538320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: from cryptography.utils import int_from_bytes Jun 27 09:14:01.581888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 09:14:01.582076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 09:14:01.582157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 09:14:01.582247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] command line args: [] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 09:14:01.582325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 09:14:01.582447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] command line args: [] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 09:14:01.582526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 09:14:01.582665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ================================================================================ {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 09:14:01.582747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 09:14:01.582885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ================================================================================ {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 09:14:01.582973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] admin_token = **** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.583140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] admin_token = **** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.583221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] config_dir = [] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.583328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.583452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] config_dir = [] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.583559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] config_source = [] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.583650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.583721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] debug = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.583787 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] config_source = [] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.583885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] debug = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.584964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.585124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] list_limit = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.585208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] list_limit = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.585385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] log_config_append = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.586055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] log_config_append = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.586190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.586316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.586443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] log_dir = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.586575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] log_dir = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.586708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] log_file = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.586836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] log_file = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.586974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.587092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.587212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.587339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.587471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.587578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.587652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.587771 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.587870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.587987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.588937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.589905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] publish_errors = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] publish_errors = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590743 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.590977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.591132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.591227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.591363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.591441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.591619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] standard_threads = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.591712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] standard_threads = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.591851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.591928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] use_journal = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] use_journal = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] use_json = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] use_json = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] use_stderr = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.592928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] use_stderr = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.593001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] use_syslog = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.593071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] use_syslog = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.593165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.593234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 09:14:01.593421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.593504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.593657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.593738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.593913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.593995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.594150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.594233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.594392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.594475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.594627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.594703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.594871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.594955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.595107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.595188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.595349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.595431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.595613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.595708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.595863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.595941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.596097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.596172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.596338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.596424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.596571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.596648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.596815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.596881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.597070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.597154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.597292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.597363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.597541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.597613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.597795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.597996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.598089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.598170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.598249 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.598320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.598492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.598574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.598749 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.598827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.598997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.599080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.599241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.599326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.599470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.599580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.599725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.599827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.599951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.600030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.600181 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.600260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.600429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.600514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.600669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.600748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.600917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.600995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.601152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.601229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.601394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.601472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.601637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.601720 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602790 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.602870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.603018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.603103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.603243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.603327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.603471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.603611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.603741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.603840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.603956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.604042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.604196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.604274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.604447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.604534 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.604648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.604738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.604878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.604974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.605112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.605191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.605349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.605431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.605591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.605678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.605820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.605904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.606052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.606146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.606289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.606382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.606523 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.606602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.606755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.606847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.606987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.607073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.607212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.607296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.607458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.607555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.607714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.607815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.607951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.608030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.608181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.608279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.608433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.608520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.608668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.608746 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.608896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.608979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.609130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.609219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.609362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.609446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.609624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.609703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.609861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.610014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.610109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.610179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.610334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.610414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.610569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.610655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.610801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.610880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.611035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.611114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.611284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.611378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.611521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.611657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.611779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.611857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.612021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.612102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.612258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.612339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.612494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.612578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.612728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.612807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.612963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.613047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.613200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.613282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.613457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.613535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.613697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.613777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.613934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.614012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.614171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.614249 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.614410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.614490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.614644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.614724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.614879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.614958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.615111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.615190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.615340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.615419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.615587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.615670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.615828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.615909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.616064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.616144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.616292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.616370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.616666 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.616757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.616830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.616901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.617026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.617105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.617261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.617342 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.617513 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.617597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.617769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.617849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.618008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.618087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.618235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] auth.external = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.618313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] auth.external = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.618465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.618543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.618754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.618865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.618986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.619067 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.619215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] auth.password = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.619294 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] auth.password = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.619441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] auth.token = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.619520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] auth.token = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.619703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.619786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.619941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.620019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.620184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.620262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.620414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.620494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.620656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.620735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.620901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.620987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.621132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.621213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.621351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] credential.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.621429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] credential.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.621600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.621687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.621846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.621928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.622091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.622177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.622324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.622404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.622564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.622646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.622798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.622878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.623057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.623149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.623308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.623397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.623559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.623654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.623822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.623901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.624121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.624205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.624368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.624448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.624631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.624711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.624882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.624962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.625134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.625213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.625370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] federation.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.625449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] federation.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.625606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.625686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.625852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.625931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.626097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.626185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.626327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.626415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.626570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.626661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.626806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.626895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.627058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.627142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.627291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.627370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.627558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.627653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.627799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.627880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.628047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.628127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.628285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.628371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.628529 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.628614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.628774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.628854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.629011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.629097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.629244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.629323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.629488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.629567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.629718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.629806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.629952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.630031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.630199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.630285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.630448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.630535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.630679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.630757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.630906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.630984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.631137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.631223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.631378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.631457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.631649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.631736 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.631900 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.631996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.632664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.632752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.632919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.632999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.633198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.633287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.633439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.633519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.633679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.633760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.633913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.634000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.634146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.634225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.634376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.634455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.634610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.634704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.634848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.634932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.635086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.635165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.635332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.635413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.635580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.635665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.635831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.635910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.636161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.636316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.636396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.636562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.636641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.636808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.636889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.637040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.637119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.637278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.637358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.637511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.637593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.637749 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.637829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.637989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.638068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.638234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.638315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.638472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.638551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.638710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.638789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.638962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.639043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.639217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.639297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.639454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.639547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.639712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.639801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.639964 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.640043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.640212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.640290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.640448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.640528 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.640684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.640763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.640919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.640997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.641149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.641231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.641384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.641463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.641641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.641728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.641869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.641948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.642114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.642192 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.642358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.642437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.642603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.642684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.642838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.642917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.643065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.643143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.643299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.643380 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.643584 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.643667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.643789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.643871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.644019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.644099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.644261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.644340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.644504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.644582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.644745 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.644825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.644988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.645066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.645230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.645309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.645458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.645538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.645698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.645776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.645933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.646014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.646168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.646248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.646402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.646481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.646637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.646715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.646877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.646957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.647120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.647199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.647351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.647429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.647622 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.647706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.648419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.648555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.648683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.648764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.649173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.649261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.649412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.649492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.649649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.649728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.649893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.649973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.650133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.650212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.650360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.650439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.650594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.650676 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.650833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] resource.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.650919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] resource.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.651078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.651157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.651324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.651405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.651579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.651662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.651834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.651930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.652092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.652177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.652328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.652408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.652574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.652661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.652807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.652886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.653045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.653123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.653278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] role.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.653366 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] role.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.653523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] role.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.653607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] role.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.653753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.653833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.653997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.654084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.654243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.654323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.654485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.654567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.654728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.654815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.654970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.655049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.655214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.655293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.655456 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.655616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.655726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.655808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.655956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.656036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.656201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.656295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.656442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.656522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.656686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.656764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.656928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.657015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.657337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.657416 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.657568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.657647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.657817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.657903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.658061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.658140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.658306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.658384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.658551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.658632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.658783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.658853 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.659034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.659113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.659282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.659362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.659518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.659620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.659768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.659846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.659999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.660081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.660230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.660309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.660476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.660554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.660728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.660808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.660972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.661050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.661208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.661288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.661442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.661522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.661672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.661750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.661905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] token.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.661983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] token.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.662139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.662218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.662383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.662461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.662617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.662697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.662870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.662950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.663111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.663189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.663347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.663426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.663642 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.663725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.663888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.663967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.664134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.664213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.664366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.664446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.664602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.664687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.664835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.664913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.665078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.665158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.665323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.665401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.665550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.665627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.665792 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.665872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.666081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.666160 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.666339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.connection = **** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.666427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.connection = **** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.666582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.666671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.666827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.666914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.667061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.667140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.667302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.667382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.667577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.667738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.667815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.667885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.668034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.668114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.668272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.668359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.668512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.668592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.668747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.668829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.668989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.669076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.669223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.669302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.669466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.669544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.669699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.669786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.669939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.670027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.670168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.670247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.670403 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.670489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.670636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.670715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 09:14:01.670787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80345) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 09:14:01.670858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80344) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 09:14:01.701715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.701890 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.701957 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.702031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.702157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.702263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.711243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.711415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.739079 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.739283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.739377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.739522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.884470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.885823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.933818 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.934051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.934169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.934384 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:01.999967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.000160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.000235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.000333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.000414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.000488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.000619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.000704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.001061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.001140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.001214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.001287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.027304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.027482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.027622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.027704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.027779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.027888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.027972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.028114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.028199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.028329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.028412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.028543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.028636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.028757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.039755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.039935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.052478 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.052669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.052782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.052894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.052980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.053117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.053203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.053332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.053417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.053542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.053625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.053750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.053843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80344) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.053960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80345) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 09:14:02.072199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.072411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 09:14:02.072496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.072640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 09:14:02.072741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.072939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 09:14:02.073022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.073160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 09:14:02.073243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.073429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.073514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.073687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.073773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.073943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.074026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.074196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.084764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.085042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.085132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.085341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.085441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.085618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.085702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.085898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.093914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.094172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.097228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.097464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.097561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.097743 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.097829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.098021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.098143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.098291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.098374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.098564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.098660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.098817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.098899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.099080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.099162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.099349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.099432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.099644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.099727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.099913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.099993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.100178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.126789 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.126981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.128436 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.128619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.128721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.128895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.134560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.134734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.134901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.135124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.135208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.135405 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.142455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.142670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.142757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.142953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.147237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.147442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.147656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.147737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.147826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.148032 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.148115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.148312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.148395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.148590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.148689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.149104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.149196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.149378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.149461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.149640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.168557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.168761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.168882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.169031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.173503 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.173598 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.177136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.177244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.177538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.177630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.183115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.183215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.186752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.186842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.190462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.190560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.190760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.190844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.191044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.191141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.191325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.191408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.191624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.191724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.191907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.191991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.210819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.210929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.211103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.211192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.211368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.211451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.211657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.211742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.219480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.219619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.221194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.221281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.221465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.221566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.224652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.224742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.224933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.225061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.225208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.225308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.225516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.225601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.225808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.226399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.239494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.239673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.239831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.239914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.240059 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.240160 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.240335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.240471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.242610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.242712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.242904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.242989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.243198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.243281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.260954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.261070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.264240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.264340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.267517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.267693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.269274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.269414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.270912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.271012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.271187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.271299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.271477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.271626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.279053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.279192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.280670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.280940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.283884 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.284072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.284188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.284364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.284547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.284754 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.284865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.285032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.294436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.294669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.294762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.294960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.295047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.295242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.302759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80345) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.303033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.303118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80344) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 09:14:02.303309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.303393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.303665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.304084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.304084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.304707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.304777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.304936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.305118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.305258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.305424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.305554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.305679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.305774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.305951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.306033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.306228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.306310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.306509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.306593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.306769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.306850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.307033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.307191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80345) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.307316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80344) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 09:14:02.344487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55eec97939c0 pid: 80345 (default app) Jun 27 09:14:02.344906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55eec97939c0 pid: 80344 (default app) Jun 27 09:14:02.347958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbbbee-74e5-4c48-9098-438f5d6e3070 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:02.348259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbbbee-74e5-4c48-9098-438f5d6e3070 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:02.348535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbbbee-74e5-4c48-9098-438f5d6e3070 None None] PATH_INFO: `/v3` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:02.353263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Mon Jun 27 09:14:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:10.995082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cf61f-3b58-49d5-9b29-4d28fbb72e38 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:10.995741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cf61f-3b58-49d5-9b29-4d28fbb72e38 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:10.995741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cf61f-3b58-49d5-9b29-4d28fbb72e38 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:11.000486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1/2] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:11.006243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:11.006717 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:11.006808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:11.076371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: INFO dbcounter [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] Registered counter for database keystone Jun 27 09:14:11.098092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG oslo_db.sqlalchemy.engines [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80345) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 09:14:11.105140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writer thread running {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 09:14:11.289022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG passlib.handlers.bcrypt [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80345) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 09:14:11.289440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG passlib.handlers.bcrypt [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80345) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 09:14:11.344941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:11.346581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:11.469642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 2/3] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 27 09:14:11.647611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cf61f-3b58-49d5-9b29-4d28fbb72e38 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:11.718119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: INFO dbcounter [None req-6a7cf61f-3b58-49d5-9b29-4d28fbb72e38 None None] Registered counter for database keystone Jun 27 09:14:11.737142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG oslo_db.sqlalchemy.engines [None req-6a7cf61f-3b58-49d5-9b29-4d28fbb72e38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80344) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 09:14:11.743910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writer thread running {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 09:14:11.870066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cf61f-3b58-49d5-9b29-4d28fbb72e38 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:11.875266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408ec14-9643-4dea-9ef1-4341dc07a855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:11.875956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408ec14-9643-4dea-9ef1-4341dc07a855 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:11.876239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408ec14-9643-4dea-9ef1-4341dc07a855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:11.876517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408ec14-9643-4dea-9ef1-4341dc07a855 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:11.885408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408ec14-9643-4dea-9ef1-4341dc07a855 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:11.938875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f408ec14-9643-4dea-9ef1-4341dc07a855 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 09:14:11.939138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408ec14-9643-4dea-9ef1-4341dc07a855 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:11.949678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 2/4] 10.222.0.64 () {58 vars in 1324 bytes} [Mon Jun 27 09:14:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 09:14:11.955712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:11.970507 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdeff2e-efff-4d0c-b937-aa298fb3588a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:11.975078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576c253-1f35-4e6f-b7fa-353adc526254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:11.976881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576c253-1f35-4e6f-b7fa-353adc526254 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:11.977166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576c253-1f35-4e6f-b7fa-353adc526254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:11.977447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576c253-1f35-4e6f-b7fa-353adc526254 admin admin] PATH_INFO: `/v3/projects/d706bacf3a904a118b091c4d80341fee` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:11.981644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576c253-1f35-4e6f-b7fa-353adc526254 admin admin] RBAC: Authorizing `identity:get_project(project_id=d706bacf3a904a118b091c4d80341fee)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:12.032686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d576c253-1f35-4e6f-b7fa-353adc526254 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 09:14:12.033325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576c253-1f35-4e6f-b7fa-353adc526254 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:12.037767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 3/5] 10.222.0.64 () {56 vars in 1207 bytes} [Mon Jun 27 09:14:11 2022] GET /identity/v3/projects/d706bacf3a904a118b091c4d80341fee => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:14.078288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac59675-cca3-4d43-ac37-b7acc51992c8 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:14.078833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac59675-cca3-4d43-ac37-b7acc51992c8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:14.078833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac59675-cca3-4d43-ac37-b7acc51992c8 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:14.082963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 3/6] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:14.088569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7528c6-76f5-4012-aa55-cbb32dbded43 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:14.088965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7528c6-76f5-4012-aa55-cbb32dbded43 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:14.089141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7528c6-76f5-4012-aa55-cbb32dbded43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:14.109102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ce7528c6-76f5-4012-aa55-cbb32dbded43 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:14.110645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ce7528c6-76f5-4012-aa55-cbb32dbded43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:14.119275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 4/7] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 09:14:14.143645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac59675-cca3-4d43-ac37-b7acc51992c8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:14.156086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac59675-cca3-4d43-ac37-b7acc51992c8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:14.160312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd99e-0736-46ff-97e6-2867fc18419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:14.160942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd99e-0736-46ff-97e6-2867fc18419b admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:14.161219 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd99e-0736-46ff-97e6-2867fc18419b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:14.161492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd99e-0736-46ff-97e6-2867fc18419b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:14.169715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bd99e-0736-46ff-97e6-2867fc18419b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:14.174794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bd99e-0736-46ff-97e6-2867fc18419b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:14.180758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 4/8] 10.222.0.64 () {58 vars in 1324 bytes} [Mon Jun 27 09:14:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:14:14.186454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7528c6-76f5-4012-aa55-cbb32dbded43 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:14.195921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7528c6-76f5-4012-aa55-cbb32dbded43 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:14.200423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b347600-5393-4c5a-b2b1-20cd09a6c16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:14.201293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b347600-5393-4c5a-b2b1-20cd09a6c16b admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:14.201579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b347600-5393-4c5a-b2b1-20cd09a6c16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:14.201928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b347600-5393-4c5a-b2b1-20cd09a6c16b admin admin] PATH_INFO: `/v3/users/e1fa6647d77b40cd8875b5b59ec1508d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:14.203277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b347600-5393-4c5a-b2b1-20cd09a6c16b admin admin] RBAC: Authorizing `identity:get_user(user_id=e1fa6647d77b40cd8875b5b59ec1508d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:14.205141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b347600-5393-4c5a-b2b1-20cd09a6c16b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:14.206790 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 5/9] 10.222.0.64 () {56 vars in 1198 bytes} [Mon Jun 27 09:14:14 2022] GET /identity/v3/users/e1fa6647d77b40cd8875b5b59ec1508d => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:16.312343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc39c6-25d8-42f4-88db-e814e398aaad None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:16.312858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc39c6-25d8-42f4-88db-e814e398aaad None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:16.312957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc39c6-25d8-42f4-88db-e814e398aaad None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:16.314265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 5/10] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:16.320257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c39306-81b0-486f-8f5c-18a9e1c2c11d None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:16.320686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c39306-81b0-486f-8f5c-18a9e1c2c11d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:16.320820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c39306-81b0-486f-8f5c-18a9e1c2c11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:16.337487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e4c39306-81b0-486f-8f5c-18a9e1c2c11d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:16.338694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e4c39306-81b0-486f-8f5c-18a9e1c2c11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:16.359856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c1c5-799c-4747-bd1e-e39e3a72888b None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:16.360311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c1c5-799c-4747-bd1e-e39e3a72888b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:16.360632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c1c5-799c-4747-bd1e-e39e3a72888b None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:16.362069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 6/11] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:16.367949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1b22d-47c3-4adc-aad9-964aee2fcd53 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:16.368891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1b22d-47c3-4adc-aad9-964aee2fcd53 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:16.369203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1b22d-47c3-4adc-aad9-964aee2fcd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:16.410035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG passlib.handlers.bcrypt [None req-5fd1b22d-47c3-4adc-aad9-964aee2fcd53 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80344) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 09:14:16.410466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG passlib.handlers.bcrypt [None req-5fd1b22d-47c3-4adc-aad9-964aee2fcd53 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80344) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 09:14:16.425354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 6/12] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:16.449802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c39306-81b0-486f-8f5c-18a9e1c2c11d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:16.456681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-5fd1b22d-47c3-4adc-aad9-964aee2fcd53 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:16.458355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5fd1b22d-47c3-4adc-aad9-964aee2fcd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:16.477652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:16.565823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c39306-81b0-486f-8f5c-18a9e1c2c11d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:16.566051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 7/13] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:16.588272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1b22d-47c3-4adc-aad9-964aee2fcd53 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:16.602448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152aa77-e750-492b-a86b-2a57f3ebcce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:16.603453 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152aa77-e750-492b-a86b-2a57f3ebcce0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:16.603860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152aa77-e750-492b-a86b-2a57f3ebcce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:16.604294 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152aa77-e750-492b-a86b-2a57f3ebcce0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:16.617199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152aa77-e750-492b-a86b-2a57f3ebcce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-5152aa77-e750-492b-a86b-2a57f3ebcce0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:16.628630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:16.629750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:16.630671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 7/14] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:14:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:16.635820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152aa77-e750-492b-a86b-2a57f3ebcce0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:16.694493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1b22d-47c3-4adc-aad9-964aee2fcd53 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:16.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:16.718724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:16.733900 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d12e07-59e9-46e3-b1f2-ae54588ca81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:16.734806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12e07-59e9-46e3-b1f2-ae54588ca81c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:16.735105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12e07-59e9-46e3-b1f2-ae54588ca81c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:16.735376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12e07-59e9-46e3-b1f2-ae54588ca81c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:16.737264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152aa77-e750-492b-a86b-2a57f3ebcce0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:16.771908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7640d-cbec-458c-9145-5cfe50094f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:16.773328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7640d-cbec-458c-9145-5cfe50094f12 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:16.773328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7640d-cbec-458c-9145-5cfe50094f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:16.773328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7640d-cbec-458c-9145-5cfe50094f12 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:16.774292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7640d-cbec-458c-9145-5cfe50094f12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:16.774601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7640d-cbec-458c-9145-5cfe50094f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:16.776053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7640d-cbec-458c-9145-5cfe50094f12 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:16.789357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 8/15] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:16.794837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7640d-cbec-458c-9145-5cfe50094f12 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:16.807169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d12e07-59e9-46e3-b1f2-ae54588ca81c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:16.808758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d12e07-59e9-46e3-b1f2-ae54588ca81c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:16.899981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7640d-cbec-458c-9145-5cfe50094f12 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:16.908657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 8/16] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 09:14:16.914895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d12e07-59e9-46e3-b1f2-ae54588ca81c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:16.917706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:16.934288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0b4a7-7340-4062-814a-353421b32ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:16.935162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0b4a7-7340-4062-814a-353421b32ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:16.935446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0b4a7-7340-4062-814a-353421b32ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:16.935753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0b4a7-7340-4062-814a-353421b32ce6 None admin] PATH_INFO: `/v3/users/e1fa6647d77b40cd8875b5b59ec1508d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:16.936983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0b4a7-7340-4062-814a-353421b32ce6 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fa6647d77b40cd8875b5b59ec1508d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:16.938507 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0b4a7-7340-4062-814a-353421b32ce6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:16.940337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 9/17] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:16 2022] GET /identity/v3/users/e1fa6647d77b40cd8875b5b59ec1508d => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:16.945694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0b4a7-7340-4062-814a-353421b32ce6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:16.961750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:17.014548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d12e07-59e9-46e3-b1f2-ae54588ca81c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:17.049542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0b4a7-7340-4062-814a-353421b32ce6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:17.050458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ccc2c-a8a7-4df6-8e8c-6df0985b9d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:17.051317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ccc2c-a8a7-4df6-8e8c-6df0985b9d7f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:17.051630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ccc2c-a8a7-4df6-8e8c-6df0985b9d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:17.051910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ccc2c-a8a7-4df6-8e8c-6df0985b9d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:17.072710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ccc2c-a8a7-4df6-8e8c-6df0985b9d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:17.074239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ccc2c-a8a7-4df6-8e8c-6df0985b9d7f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:17.089673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff171f-6593-4a05-921e-303223095c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:17.090473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff171f-6593-4a05-921e-303223095c2e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:17.090756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff171f-6593-4a05-921e-303223095c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:17.091038 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff171f-6593-4a05-921e-303223095c2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:17.092263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff171f-6593-4a05-921e-303223095c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:17.093708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff171f-6593-4a05-921e-303223095c2e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:17.095332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 10/18] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-707ccc2c-a8a7-4df6-8e8c-6df0985b9d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:14:17.095435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:17.096481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:17.097283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:14:17.097283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:14:17.097283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:14:17.097283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:14:17.097283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 9/19] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:14:17.102359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff171f-6593-4a05-921e-303223095c2e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:17.102653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ccc2c-a8a7-4df6-8e8c-6df0985b9d7f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:17.158623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:17.184194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:17.203888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff171f-6593-4a05-921e-303223095c2e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:17.212541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ccc2c-a8a7-4df6-8e8c-6df0985b9d7f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:17.237842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17431d2-ee27-45a9-8c74-17038e23102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:17.238499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17431d2-ee27-45a9-8c74-17038e23102c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:17.238779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17431d2-ee27-45a9-8c74-17038e23102c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:17.239055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17431d2-ee27-45a9-8c74-17038e23102c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:17.239913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17431d2-ee27-45a9-8c74-17038e23102c None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.domain.id=default, user.id=e1fa6647d77b40cd8875b5b59ec1508d) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:17.240224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17431d2-ee27-45a9-8c74-17038e23102c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:17.241689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17431d2-ee27-45a9-8c74-17038e23102c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:17.245941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793119c-d707-4c95-9f0c-7d8e2679b6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:17.246618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793119c-d707-4c95-9f0c-7d8e2679b6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:17.246904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793119c-d707-4c95-9f0c-7d8e2679b6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:17.247185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793119c-d707-4c95-9f0c-7d8e2679b6b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:17.247956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793119c-d707-4c95-9f0c-7d8e2679b6b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:17.248269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793119c-d707-4c95-9f0c-7d8e2679b6b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:17.249035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 11/20] 10.222.0.64 () {56 vars in 1322 bytes} [Mon Jun 27 09:14:17 2022] GET /identity/v3/role_assignments?user.id=e1fa6647d77b40cd8875b5b59ec1508d&scope.domain.id=default&role.id=03b724e03f704f2ab338b64025a3db6d => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:17.249712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793119c-d707-4c95-9f0c-7d8e2679b6b4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:17.273966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 10/21] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:19.564752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7cb94-b295-4d73-96ca-c79c1432fe53 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:19.565289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7cb94-b295-4d73-96ca-c79c1432fe53 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:19.565289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7cb94-b295-4d73-96ca-c79c1432fe53 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:19.566635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 12/22] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:19.585638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5bd8b-d03f-4c63-b952-e16a82ccf42d None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:19.586136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5bd8b-d03f-4c63-b952-e16a82ccf42d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:19.586225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5bd8b-d03f-4c63-b952-e16a82ccf42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:19.612525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-97b5bd8b-d03f-4c63-b952-e16a82ccf42d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:19.613732 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-97b5bd8b-d03f-4c63-b952-e16a82ccf42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:19.697862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 11/23] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:19.732693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7cb94-b295-4d73-96ca-c79c1432fe53 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:19.836106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dc678-ee76-4516-94dc-29f2f32adcc9 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:19.836290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dc678-ee76-4516-94dc-29f2f32adcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:19.836567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dc678-ee76-4516-94dc-29f2f32adcc9 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:19.837975 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 12/24] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:19.843853 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc4196-8991-4a69-bcc1-a197df27b614 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:19.844131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc4196-8991-4a69-bcc1-a197df27b614 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:19.844415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc4196-8991-4a69-bcc1-a197df27b614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:19.880670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-72dc4196-8991-4a69-bcc1-a197df27b614 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:19.882466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-72dc4196-8991-4a69-bcc1-a197df27b614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:19.901521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7cb94-b295-4d73-96ca-c79c1432fe53 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:19.951455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:19.969394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9dd77-d9d2-4536-a6ac-73adc52abc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:19.970212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9dd77-d9d2-4536-a6ac-73adc52abc81 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:19.970502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9dd77-d9d2-4536-a6ac-73adc52abc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:19.970780 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9dd77-d9d2-4536-a6ac-73adc52abc81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:19.990064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 13/25] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:19.997070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9dd77-d9d2-4536-a6ac-73adc52abc81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-edd9dd77-d9d2-4536-a6ac-73adc52abc81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:20.012429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:20.014625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:20.016576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 13/26] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:20.023560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc4196-8991-4a69-bcc1-a197df27b614 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:20.047771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9dd77-d9d2-4536-a6ac-73adc52abc81 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:20.055674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:20.125359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc4196-8991-4a69-bcc1-a197df27b614 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:20.166125 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329bf32-f4bb-4d53-a498-ca3cf47a0800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:20.166807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329bf32-f4bb-4d53-a498-ca3cf47a0800 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.167067 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329bf32-f4bb-4d53-a498-ca3cf47a0800 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.167342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329bf32-f4bb-4d53-a498-ca3cf47a0800 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.168136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329bf32-f4bb-4d53-a498-ca3cf47a0800 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:20.168440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329bf32-f4bb-4d53-a498-ca3cf47a0800 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:20.169827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329bf32-f4bb-4d53-a498-ca3cf47a0800 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:20.196693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 14/27] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:20.198049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471cad9-b776-42d8-8804-8657f7589392 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.198339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471cad9-b776-42d8-8804-8657f7589392 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.198613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471cad9-b776-42d8-8804-8657f7589392 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.202507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 15/28] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:20.203266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471cad9-b776-42d8-8804-8657f7589392 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:20.246344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9dd77-d9d2-4536-a6ac-73adc52abc81 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:20.291923 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:20.307010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141572e-38df-485f-8c39-288e32b399cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:20.310864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141572e-38df-485f-8c39-288e32b399cc None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.310864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141572e-38df-485f-8c39-288e32b399cc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.310864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141572e-38df-485f-8c39-288e32b399cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.334122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141572e-38df-485f-8c39-288e32b399cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:20.336714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141572e-38df-485f-8c39-288e32b399cc None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-f141572e-38df-485f-8c39-288e32b399cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:14:20.368126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:20.368992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:20.369814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:14:20.369814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:14:20.369814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:14:20.369814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:20.369814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 14/29] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:14:20.375618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290095d-45bc-4441-b14d-5edf3de7297c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.375618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290095d-45bc-4441-b14d-5edf3de7297c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.375618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290095d-45bc-4441-b14d-5edf3de7297c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.409964 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4290095d-45bc-4441-b14d-5edf3de7297c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:20.413500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4290095d-45bc-4441-b14d-5edf3de7297c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:20.429437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471cad9-b776-42d8-8804-8657f7589392 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:20.502628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:20.505367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c0aab-7cf4-4f10-9905-21a60ecbc2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:20.506269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c0aab-7cf4-4f10-9905-21a60ecbc2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.507080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c0aab-7cf4-4f10-9905-21a60ecbc2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.507354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c0aab-7cf4-4f10-9905-21a60ecbc2d4 None admin] PATH_INFO: `/v3/users/e1fa6647d77b40cd8875b5b59ec1508d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.511566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c0aab-7cf4-4f10-9905-21a60ecbc2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fa6647d77b40cd8875b5b59ec1508d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:20.513398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c0aab-7cf4-4f10-9905-21a60ecbc2d4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:20.516610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 16/30] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:20 2022] GET /identity/v3/users/e1fa6647d77b40cd8875b5b59ec1508d => generated 247 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:20.516610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c0aab-7cf4-4f10-9905-21a60ecbc2d4 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:20.525201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 15/31] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:20.526828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ca16f-8411-4b5d-ba8a-905528b440c2 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.527179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ca16f-8411-4b5d-ba8a-905528b440c2 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.527465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ca16f-8411-4b5d-ba8a-905528b440c2 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.529285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 16/32] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:20.530259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ca16f-8411-4b5d-ba8a-905528b440c2 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:20.623259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c0aab-7cf4-4f10-9905-21a60ecbc2d4 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:20.636155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:20.647452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ca16f-8411-4b5d-ba8a-905528b440c2 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:20.658885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206a9a5-09d2-411e-b067-f33c4f69fca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:20.659656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206a9a5-09d2-411e-b067-f33c4f69fca6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.659976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206a9a5-09d2-411e-b067-f33c4f69fca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.660342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206a9a5-09d2-411e-b067-f33c4f69fca6 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.661516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206a9a5-09d2-411e-b067-f33c4f69fca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:20.661516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206a9a5-09d2-411e-b067-f33c4f69fca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:20.663003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206a9a5-09d2-411e-b067-f33c4f69fca6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:20.682146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 17/33] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:20.682146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef656d1-4af3-4bc1-af49-4a4ce75e39d8 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.682146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef656d1-4af3-4bc1-af49-4a4ce75e39d8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.682146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef656d1-4af3-4bc1-af49-4a4ce75e39d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.698085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffa497-2660-47a1-bacc-f674f35c8462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:20.698671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-3ef656d1-4af3-4bc1-af49-4a4ce75e39d8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:20.699563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffa497-2660-47a1-bacc-f674f35c8462 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.700037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3ef656d1-4af3-4bc1-af49-4a4ce75e39d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:20.700256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffa497-2660-47a1-bacc-f674f35c8462 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.700734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffa497-2660-47a1-bacc-f674f35c8462 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.702224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffa497-2660-47a1-bacc-f674f35c8462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:20.703995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdffa497-2660-47a1-bacc-f674f35c8462 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:20.705953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 17/34] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:20.707039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffa497-2660-47a1-bacc-f674f35c8462 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:20.771815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:20.781289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 18/35] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:20.782151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef656d1-4af3-4bc1-af49-4a4ce75e39d8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:20.822824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffa497-2660-47a1-bacc-f674f35c8462 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:20.860712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269981a0-af25-4b1f-97dc-e6794196b367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:20.861734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269981a0-af25-4b1f-97dc-e6794196b367 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.862221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269981a0-af25-4b1f-97dc-e6794196b367 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.862726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269981a0-af25-4b1f-97dc-e6794196b367 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.863740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269981a0-af25-4b1f-97dc-e6794196b367 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:20.865334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269981a0-af25-4b1f-97dc-e6794196b367 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:20.884776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:20.887888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef656d1-4af3-4bc1-af49-4a4ce75e39d8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:20.923181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f43aa4-d362-4f48-b780-12a3c7b26f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:20.923903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f43aa4-d362-4f48-b780-12a3c7b26f79 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:20.924185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f43aa4-d362-4f48-b780-12a3c7b26f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:20.924455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f43aa4-d362-4f48-b780-12a3c7b26f79 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:20.925180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f43aa4-d362-4f48-b780-12a3c7b26f79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:20.926646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f43aa4-d362-4f48-b780-12a3c7b26f79 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:21.088785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 18/36] 10.222.0.64 () {60 vars in 1131 bytes} [Mon Jun 27 09:14:20 2022] POST /identity/v3/roles => generated 219 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:14:21.090153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269981a0-af25-4b1f-97dc-e6794196b367 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:21.182645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 19/37] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:14:20 2022] POST /identity/v3/projects => generated 301 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:14:21.184138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f43aa4-d362-4f48-b780-12a3c7b26f79 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:21.214773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269981a0-af25-4b1f-97dc-e6794196b367 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:21.231762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:21.254797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ec845-f532-4497-96d6-c193ccf84f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:21.255722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ec845-f532-4497-96d6-c193ccf84f3d None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:21.256021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ec845-f532-4497-96d6-c193ccf84f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:21.256314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ec845-f532-4497-96d6-c193ccf84f3d None admin] PATH_INFO: `/v3/domains/default/users/e1fa6647d77b40cd8875b5b59ec1508d/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:21.259259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ec845-f532-4497-96d6-c193ccf84f3d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e1fa6647d77b40cd8875b5b59ec1508d, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:21.260925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ec845-f532-4497-96d6-c193ccf84f3d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:21.328322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f43aa4-d362-4f48-b780-12a3c7b26f79 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:21.379914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff1649-fe3e-4e03-8f45-4fb46a1acf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:21.380656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff1649-fe3e-4e03-8f45-4fb46a1acf68 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:21.380939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff1649-fe3e-4e03-8f45-4fb46a1acf68 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:21.381346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff1649-fe3e-4e03-8f45-4fb46a1acf68 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:21.382176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff1649-fe3e-4e03-8f45-4fb46a1acf68 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:21.383802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff1649-fe3e-4e03-8f45-4fb46a1acf68 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:21.393658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 19/38] 10.222.0.64 () {58 vars in 1361 bytes} [Mon Jun 27 09:14:21 2022] PUT /identity/v3/domains/default/users/e1fa6647d77b40cd8875b5b59ec1508d/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:14:21.395234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73feb86-33dd-41ff-960e-e57771bf358d None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:21.395547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73feb86-33dd-41ff-960e-e57771bf358d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:21.395833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73feb86-33dd-41ff-960e-e57771bf358d None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:21.397905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 20/39] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:21.404655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcbe23-0de7-4997-bfb5-62487c324edd None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:21.404963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcbe23-0de7-4997-bfb5-62487c324edd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:21.405243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcbe23-0de7-4997-bfb5-62487c324edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:21.427329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4cfcbe23-0de7-4997-bfb5-62487c324edd None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:21.429013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4cfcbe23-0de7-4997-bfb5-62487c324edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:21.483092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 20/40] 10.222.0.64 () {60 vars in 1131 bytes} [Mon Jun 27 09:14:21 2022] POST /identity/v3/roles => generated 225 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:14:21.485310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c66be-7853-4a0a-8946-21bf54fc0258 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:21.485721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c66be-7853-4a0a-8946-21bf54fc0258 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:21.486176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c66be-7853-4a0a-8946-21bf54fc0258 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:21.488544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 21/41] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:21.494777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99891f6c-a0a7-43cc-8809-b4f8cf45c3b5 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:21.495242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99891f6c-a0a7-43cc-8809-b4f8cf45c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:21.495585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99891f6c-a0a7-43cc-8809-b4f8cf45c3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:21.519811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-99891f6c-a0a7-43cc-8809-b4f8cf45c3b5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:21.521775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-99891f6c-a0a7-43cc-8809-b4f8cf45c3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:21.525579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:21.525754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 21/42] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:21.559093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcbe23-0de7-4997-bfb5-62487c324edd None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:21.619214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 22/43] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:21.643399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99891f6c-a0a7-43cc-8809-b4f8cf45c3b5 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:21.689830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcbe23-0de7-4997-bfb5-62487c324edd None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:21.693530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:21.741727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d515c-b06b-4e89-b92d-1d3310710430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:21.742801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d515c-b06b-4e89-b92d-1d3310710430 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:21.743332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d515c-b06b-4e89-b92d-1d3310710430 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:21.743870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d515c-b06b-4e89-b92d-1d3310710430 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:21.744878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d515c-b06b-4e89-b92d-1d3310710430 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:21.746633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d515c-b06b-4e89-b92d-1d3310710430 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:21.795274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99891f6c-a0a7-43cc-8809-b4f8cf45c3b5 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:21.834076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bef2b-c484-4b6e-b153-966525bff8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:21.835573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bef2b-c484-4b6e-b153-966525bff8da None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:21.835846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bef2b-c484-4b6e-b153-966525bff8da None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:21.836129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bef2b-c484-4b6e-b153-966525bff8da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:21.837365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bef2b-c484-4b6e-b153-966525bff8da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:21.838858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bef2b-c484-4b6e-b153-966525bff8da None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:21.840555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 23/44] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:21.846028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bef2b-c484-4b6e-b153-966525bff8da None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:21.854916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:21.880724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 22/45] 10.222.0.64 () {60 vars in 1131 bytes} [Mon Jun 27 09:14:21 2022] POST /identity/v3/roles => generated 223 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:14:21.967798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bef2b-c484-4b6e-b153-966525bff8da None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:22.009252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b508d0b-4306-4807-bd23-4d6a367d6003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:22.010054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b508d0b-4306-4807-bd23-4d6a367d6003 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:22.010339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b508d0b-4306-4807-bd23-4d6a367d6003 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:22.010617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b508d0b-4306-4807-bd23-4d6a367d6003 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:22.011349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b508d0b-4306-4807-bd23-4d6a367d6003 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:22.012886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b508d0b-4306-4807-bd23-4d6a367d6003 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:22.103579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 24/46] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:14:21 2022] POST /identity/v3/projects => generated 312 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:14:23.499122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840fb3d-cb60-423e-8307-8918cd416c37 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:23.499677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840fb3d-cb60-423e-8307-8918cd416c37 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:23.499781 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840fb3d-cb60-423e-8307-8918cd416c37 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:23.501133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 23/47] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:23.507437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3b546-52e0-444b-ab4f-cbbbd26a705a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:23.507955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3b546-52e0-444b-ab4f-cbbbd26a705a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:23.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3b546-52e0-444b-ab4f-cbbbd26a705a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:23.525392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f7a3b546-52e0-444b-ab4f-cbbbd26a705a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:23.526497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f7a3b546-52e0-444b-ab4f-cbbbd26a705a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:23.580332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:23.593829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 25/48] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:23.617351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840fb3d-cb60-423e-8307-8918cd416c37 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:23.834922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840fb3d-cb60-423e-8307-8918cd416c37 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:23.871914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb85df-aed4-4763-8663-c72a3cf96f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:23.872753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb85df-aed4-4763-8663-c72a3cf96f1e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:23.873037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb85df-aed4-4763-8663-c72a3cf96f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:23.873321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb85df-aed4-4763-8663-c72a3cf96f1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:23.886442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb85df-aed4-4763-8663-c72a3cf96f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-abbb85df-aed4-4763-8663-c72a3cf96f1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:23.894621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:23.895700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:23.896445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 24/49] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:14:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:23.902198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3b546-52e0-444b-ab4f-cbbbd26a705a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:24.111945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3b546-52e0-444b-ab4f-cbbbd26a705a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:24.146569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e992f5-5c40-4549-8d63-e802b3ae1273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:24.147291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e992f5-5c40-4549-8d63-e802b3ae1273 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:24.147621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e992f5-5c40-4549-8d63-e802b3ae1273 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:24.147910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e992f5-5c40-4549-8d63-e802b3ae1273 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:24.148679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e992f5-5c40-4549-8d63-e802b3ae1273 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:24.148994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e992f5-5c40-4549-8d63-e802b3ae1273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:24.150651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e992f5-5c40-4549-8d63-e802b3ae1273 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:24.164524 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 26/50] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:24.165734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb85df-aed4-4763-8663-c72a3cf96f1e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:24.186249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c76692-f2d3-4e58-888b-a01dd5f87568 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:24.188826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c76692-f2d3-4e58-888b-a01dd5f87568 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:24.188826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c76692-f2d3-4e58-888b-a01dd5f87568 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:24.188826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 27/51] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:24.193391 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:24.193520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ecb87-fc0c-4163-9543-cea9ea4bc7cd None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:24.193719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ecb87-fc0c-4163-9543-cea9ea4bc7cd None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:24.194001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ecb87-fc0c-4163-9543-cea9ea4bc7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:24.210675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-0f6ecb87-fc0c-4163-9543-cea9ea4bc7cd None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:24.211952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0f6ecb87-fc0c-4163-9543-cea9ea4bc7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:24.259648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:24.271999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb85df-aed4-4763-8663-c72a3cf96f1e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:24.283467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 28/52] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:24.310332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ecb87-fc0c-4163-9543-cea9ea4bc7cd None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:24.310792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8b14a-1f0f-47ee-b98f-4a49452640fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:24.311768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8b14a-1f0f-47ee-b98f-4a49452640fe None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:24.312023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8b14a-1f0f-47ee-b98f-4a49452640fe None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:24.312302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8b14a-1f0f-47ee-b98f-4a49452640fe None admin] PATH_INFO: `/v3/users/e1fa6647d77b40cd8875b5b59ec1508d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:24.313609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d8b14a-1f0f-47ee-b98f-4a49452640fe None admin] RBAC: Authorizing `identity:get_user(user_id=e1fa6647d77b40cd8875b5b59ec1508d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:24.315162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d8b14a-1f0f-47ee-b98f-4a49452640fe None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:24.316819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 25/53] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:24 2022] GET /identity/v3/users/e1fa6647d77b40cd8875b5b59ec1508d => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:24.322763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8b14a-1f0f-47ee-b98f-4a49452640fe None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:24.413978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ecb87-fc0c-4163-9543-cea9ea4bc7cd None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:24.418461 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:24.427552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8b14a-1f0f-47ee-b98f-4a49452640fe None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:24.450845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b5a64-e5bc-499d-9373-2b1f8101d5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:24.451681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5a64-e5bc-499d-9373-2b1f8101d5ba None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:24.451965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5a64-e5bc-499d-9373-2b1f8101d5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:24.452281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b5a64-e5bc-499d-9373-2b1f8101d5ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:24.453429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b5a64-e5bc-499d-9373-2b1f8101d5ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:24.454887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b5a64-e5bc-499d-9373-2b1f8101d5ba None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:24.456576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 29/54] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:24.462988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee171f1-242e-4a9b-a807-57b6339ff475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:24.463604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b5a64-e5bc-499d-9373-2b1f8101d5ba None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:24.463863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee171f1-242e-4a9b-a807-57b6339ff475 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:24.464153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee171f1-242e-4a9b-a807-57b6339ff475 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:24.464435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee171f1-242e-4a9b-a807-57b6339ff475 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:24.465650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee171f1-242e-4a9b-a807-57b6339ff475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:24.467165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee171f1-242e-4a9b-a807-57b6339ff475 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:24.469074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 26/55] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:24.474744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee171f1-242e-4a9b-a807-57b6339ff475 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:24.516407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:24.571505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b5a64-e5bc-499d-9373-2b1f8101d5ba None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:24.577793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee171f1-242e-4a9b-a807-57b6339ff475 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:24.606988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0445733-1df1-4403-afbb-63177ee96cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:24.607780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0445733-1df1-4403-afbb-63177ee96cfe None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:24.608065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0445733-1df1-4403-afbb-63177ee96cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:24.608342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0445733-1df1-4403-afbb-63177ee96cfe None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:24.609090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0445733-1df1-4403-afbb-63177ee96cfe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:24.610561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0445733-1df1-4403-afbb-63177ee96cfe None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:24.613318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223598f-8145-4651-8c0e-948993f9d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:24.614157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223598f-8145-4651-8c0e-948993f9d179 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:24.614446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223598f-8145-4651-8c0e-948993f9d179 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:24.615058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223598f-8145-4651-8c0e-948993f9d179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:24.616060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223598f-8145-4651-8c0e-948993f9d179 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.domain.id=default, user.id=e1fa6647d77b40cd8875b5b59ec1508d) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:24.616376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223598f-8145-4651-8c0e-948993f9d179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:24.617846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223598f-8145-4651-8c0e-948993f9d179 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:24.625196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 27/56] 10.222.0.64 () {56 vars in 1322 bytes} [Mon Jun 27 09:14:24 2022] GET /identity/v3/role_assignments?user.id=e1fa6647d77b40cd8875b5b59ec1508d&scope.domain.id=default&role.id=03b724e03f704f2ab338b64025a3db6d => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:24.691966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 30/57] 10.222.0.64 () {60 vars in 1140 bytes} [Mon Jun 27 09:14:24 2022] POST /identity/v3/projects => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:14:26.734930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31bab9-8522-44a9-a5fd-3ecf0158b573 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:26.735439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31bab9-8522-44a9-a5fd-3ecf0158b573 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:26.735547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31bab9-8522-44a9-a5fd-3ecf0158b573 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:26.736934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 28/58] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:26.742806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47569337-b33b-4acc-b5db-721c08346d3e None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:26.743220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47569337-b33b-4acc-b5db-721c08346d3e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:26.743364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47569337-b33b-4acc-b5db-721c08346d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:26.760273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-47569337-b33b-4acc-b5db-721c08346d3e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:26.761463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-47569337-b33b-4acc-b5db-721c08346d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:26.826445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 31/59] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:26.850866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31bab9-8522-44a9-a5fd-3ecf0158b573 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:26.876641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:26.953400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31bab9-8522-44a9-a5fd-3ecf0158b573 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:26.987422 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff6582-3b86-4939-b953-d93732622ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:26.988166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff6582-3b86-4939-b953-d93732622ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:26.988454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff6582-3b86-4939-b953-d93732622ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:26.988736 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff6582-3b86-4939-b953-d93732622ab5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:26.989953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff6582-3b86-4939-b953-d93732622ab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:26.992093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff6582-3b86-4939-b953-d93732622ab5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:26.993747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 29/60] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:26.999324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47569337-b33b-4acc-b5db-721c08346d3e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:27.029660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:27.098578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47569337-b33b-4acc-b5db-721c08346d3e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:27.133350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9becb3-cb68-4103-aec7-d2a416838419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:27.134028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9becb3-cb68-4103-aec7-d2a416838419 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:27.134309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9becb3-cb68-4103-aec7-d2a416838419 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:27.134717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9becb3-cb68-4103-aec7-d2a416838419 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:27.135488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9becb3-cb68-4103-aec7-d2a416838419 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:27.136792 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9becb3-cb68-4103-aec7-d2a416838419 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:27.261830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 32/61] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:14:26 2022] POST /identity/v3/users => generated 275 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:14:30.514245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d28717-5cf9-44cf-af4f-d53244beb29b None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:30.515313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d28717-5cf9-44cf-af4f-d53244beb29b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:30.515846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d28717-5cf9-44cf-af4f-d53244beb29b None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:30.517810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 30/62] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:30.523412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ce77d-f68a-494b-a361-65db81a7bc75 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:30.524022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ce77d-f68a-494b-a361-65db81a7bc75 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:30.524022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ce77d-f68a-494b-a361-65db81a7bc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:30.548618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-6a3ce77d-f68a-494b-a361-65db81a7bc75 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:30.549849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6a3ce77d-f68a-494b-a361-65db81a7bc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:30.604237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dfe5f-b9af-4cfe-b154-06d911dd10d8 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:30.604885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dfe5f-b9af-4cfe-b154-06d911dd10d8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:30.605375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dfe5f-b9af-4cfe-b154-06d911dd10d8 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:30.606978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 31/63] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:30.612874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705dfe4-35bb-41f6-94b5-92bec80ca25f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:30.613480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705dfe4-35bb-41f6-94b5-92bec80ca25f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:30.613988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705dfe4-35bb-41f6-94b5-92bec80ca25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:30.644150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-0705dfe4-35bb-41f6-94b5-92bec80ca25f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:30.646308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-0705dfe4-35bb-41f6-94b5-92bec80ca25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:30.647981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 33/64] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:30.695026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ce77d-f68a-494b-a361-65db81a7bc75 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:30.732042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:30.782206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 32/65] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:30.807303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e219f-60f2-4612-81a1-aa52e81232be None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:30.807663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e219f-60f2-4612-81a1-aa52e81232be None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:30.807906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e219f-60f2-4612-81a1-aa52e81232be None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:30.809612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 33/66] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:30.810620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e219f-60f2-4612-81a1-aa52e81232be None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:30.843943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:30.872193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ce77d-f68a-494b-a361-65db81a7bc75 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:30.924587 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049def2d-4c05-40d1-91df-707d373fb11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:30.926059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049def2d-4c05-40d1-91df-707d373fb11f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:30.926838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049def2d-4c05-40d1-91df-707d373fb11f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:30.927755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049def2d-4c05-40d1-91df-707d373fb11f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:30.939208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e219f-60f2-4612-81a1-aa52e81232be None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:30.944833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049def2d-4c05-40d1-91df-707d373fb11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-049def2d-4c05-40d1-91df-707d373fb11f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:30.957298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:30.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:30.959434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:14:30.963607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 34/67] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:30.965379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411b93b-3455-40e6-be4a-ab3768030f39 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:30.965674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411b93b-3455-40e6-be4a-ab3768030f39 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:30.965955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411b93b-3455-40e6-be4a-ab3768030f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:30.989376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f18a6-e74c-4003-9414-80bd2429a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:30.990229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f18a6-e74c-4003-9414-80bd2429a65d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:30.991266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f18a6-e74c-4003-9414-80bd2429a65d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:30.991481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f18a6-e74c-4003-9414-80bd2429a65d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:30.996320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d411b93b-3455-40e6-be4a-ab3768030f39 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:30.998318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d411b93b-3455-40e6-be4a-ab3768030f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:31.005101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f18a6-e74c-4003-9414-80bd2429a65d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-0c9f18a6-e74c-4003-9414-80bd2429a65d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:31.013679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:31.014656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:31.015424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 34/68] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:14:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:31.016985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f18a6-e74c-4003-9414-80bd2429a65d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.088932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 35/69] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:31.090673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf518433-cdc9-429b-86c2-3c1d90d7a415 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.090966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf518433-cdc9-429b-86c2-3c1d90d7a415 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.091250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf518433-cdc9-429b-86c2-3c1d90d7a415 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.096649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 36/70] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:31.096649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf518433-cdc9-429b-86c2-3c1d90d7a415 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.105676 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:31.132138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:31.153444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f18a6-e74c-4003-9414-80bd2429a65d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:31.190769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b43ca5-9965-455e-bea8-fd5e87e04d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:31.191452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b43ca5-9965-455e-bea8-fd5e87e04d3a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.191797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b43ca5-9965-455e-bea8-fd5e87e04d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.192085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b43ca5-9965-455e-bea8-fd5e87e04d3a None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.192845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b43ca5-9965-455e-bea8-fd5e87e04d3a None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:31.193155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b43ca5-9965-455e-bea8-fd5e87e04d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:31.194585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b43ca5-9965-455e-bea8-fd5e87e04d3a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:31.203574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 35/71] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:31.204882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1a2ca-a39b-4441-a459-9df468dcd560 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.205181 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1a2ca-a39b-4441-a459-9df468dcd560 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.205466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1a2ca-a39b-4441-a459-9df468dcd560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.214217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf518433-cdc9-429b-86c2-3c1d90d7a415 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:31.223706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3ac1a2ca-a39b-4441-a459-9df468dcd560 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:31.225553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3ac1a2ca-a39b-4441-a459-9df468dcd560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:31.250919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af0753-26a2-4550-b975-c1f32945b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:31.251643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af0753-26a2-4550-b975-c1f32945b422 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.251931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af0753-26a2-4550-b975-c1f32945b422 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.252220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af0753-26a2-4550-b975-c1f32945b422 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.252991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af0753-26a2-4550-b975-c1f32945b422 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:31.253307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af0753-26a2-4550-b975-c1f32945b422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:31.254747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af0753-26a2-4550-b975-c1f32945b422 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:31.264656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 37/72] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:31.265619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af0753-26a2-4550-b975-c1f32945b422 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.298107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 36/73] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:31.299478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a110f6-1689-4aaf-814e-566a63ac6e40 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.299809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a110f6-1689-4aaf-814e-566a63ac6e40 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.300095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a110f6-1689-4aaf-814e-566a63ac6e40 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.301592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 37/74] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:31.302171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a110f6-1689-4aaf-814e-566a63ac6e40 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.349038 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:31.382124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:31.384557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af0753-26a2-4550-b975-c1f32945b422 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:31.410096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a110f6-1689-4aaf-814e-566a63ac6e40 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:31.420597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c165dd-04fa-4f57-8734-5153dd775d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:31.421428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c165dd-04fa-4f57-8734-5153dd775d59 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.421741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c165dd-04fa-4f57-8734-5153dd775d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.422024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c165dd-04fa-4f57-8734-5153dd775d59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.432886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c165dd-04fa-4f57-8734-5153dd775d59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-e6c165dd-04fa-4f57-8734-5153dd775d59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:31.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:31.442236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:14:31.442991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 38/75] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:31.444206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c165dd-04fa-4f57-8734-5153dd775d59 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.450797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77d849-9f7f-49b9-97db-a3c8cab4ec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:31.451738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77d849-9f7f-49b9-97db-a3c8cab4ec1b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.452026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77d849-9f7f-49b9-97db-a3c8cab4ec1b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.452306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77d849-9f7f-49b9-97db-a3c8cab4ec1b None admin] PATH_INFO: `/v3/users/8390e06ba7334f4d8e1afbf76a9b3591` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.467337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77d849-9f7f-49b9-97db-a3c8cab4ec1b None admin] RBAC: Authorizing `identity:get_user(user_id=8390e06ba7334f4d8e1afbf76a9b3591)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:31.468857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77d849-9f7f-49b9-97db-a3c8cab4ec1b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:31.470630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 38/76] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/users/8390e06ba7334f4d8e1afbf76a9b3591 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:31.475359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10baa7-bb1e-4a11-93fb-227e118b61df None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.475359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10baa7-bb1e-4a11-93fb-227e118b61df None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.475359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10baa7-bb1e-4a11-93fb-227e118b61df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.491811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8d10baa7-bb1e-4a11-93fb-227e118b61df None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:31.493053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8d10baa7-bb1e-4a11-93fb-227e118b61df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:31.547236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c165dd-04fa-4f57-8734-5153dd775d59 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:31.561199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 39/77] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:31.562074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10baa7-bb1e-4a11-93fb-227e118b61df None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.574639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:31.582327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b5a16-4ab2-42b5-9ffd-25a6a2ee5364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:31.583230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b5a16-4ab2-42b5-9ffd-25a6a2ee5364 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.583516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b5a16-4ab2-42b5-9ffd-25a6a2ee5364 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.583825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b5a16-4ab2-42b5-9ffd-25a6a2ee5364 None admin] PATH_INFO: `/v3/users/e1fa6647d77b40cd8875b5b59ec1508d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.585886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b5a16-4ab2-42b5-9ffd-25a6a2ee5364 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fa6647d77b40cd8875b5b59ec1508d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:31.587476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b5a16-4ab2-42b5-9ffd-25a6a2ee5364 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:31.589298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 39/78] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/users/e1fa6647d77b40cd8875b5b59ec1508d => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:31.590092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b5a16-4ab2-42b5-9ffd-25a6a2ee5364 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.641450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:31.680565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10baa7-bb1e-4a11-93fb-227e118b61df None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:31.693186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b5a16-4ab2-42b5-9ffd-25a6a2ee5364 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:31.718950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a41a6-c9d1-485d-b7b1-640a66b9f4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:31.720058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a41a6-c9d1-485d-b7b1-640a66b9f4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.720342 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a41a6-c9d1-485d-b7b1-640a66b9f4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.720618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a41a6-c9d1-485d-b7b1-640a66b9f4c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.730090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d800ea-b879-4d68-b317-12bb3bcb1819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:31.730782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d800ea-b879-4d68-b317-12bb3bcb1819 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.731068 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d800ea-b879-4d68-b317-12bb3bcb1819 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.731346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d800ea-b879-4d68-b317-12bb3bcb1819 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.731454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a41a6-c9d1-485d-b7b1-640a66b9f4c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:31.732193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d800ea-b879-4d68-b317-12bb3bcb1819 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:31.732510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d800ea-b879-4d68-b317-12bb3bcb1819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:31.734010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d800ea-b879-4d68-b317-12bb3bcb1819 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-9c2a41a6-c9d1-485d-b7b1-640a66b9f4c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:31.739415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:31.740510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 40/79] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:31.741468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a41a6-c9d1-485d-b7b1-640a66b9f4c0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.747262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 40/80] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:31.748204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d800ea-b879-4d68-b317-12bb3bcb1819 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.810663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:31.852265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a41a6-c9d1-485d-b7b1-640a66b9f4c0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:31.854543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d800ea-b879-4d68-b317-12bb3bcb1819 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:31.876969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:31.889390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c440a3-04cd-4449-b981-faab450830c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:31.895698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c440a3-04cd-4449-b981-faab450830c1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.895698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c440a3-04cd-4449-b981-faab450830c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.895912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018b127-4778-475a-81d4-a300f6609294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:31.896000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c440a3-04cd-4449-b981-faab450830c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.896079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018b127-4778-475a-81d4-a300f6609294 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:31.896079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018b127-4778-475a-81d4-a300f6609294 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:31.896079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018b127-4778-475a-81d4-a300f6609294 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:31.896079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018b127-4778-475a-81d4-a300f6609294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:31.896079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018b127-4778-475a-81d4-a300f6609294 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:31.896483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 41/81] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:31.897223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018b127-4778-475a-81d4-a300f6609294 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:31.963115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c440a3-04cd-4449-b981-faab450830c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:31.964676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c440a3-04cd-4449-b981-faab450830c1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:31.999717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018b127-4778-475a-81d4-a300f6609294 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:32.025750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:32.036919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f4afb-5084-424f-9944-68808f703673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:32.037640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f4afb-5084-424f-9944-68808f703673 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:32.037932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f4afb-5084-424f-9944-68808f703673 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:32.038218 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f4afb-5084-424f-9944-68808f703673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:32.068543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 41/82] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:32.069997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c440a3-04cd-4449-b981-faab450830c1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:32.096039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:32.109888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f4afb-5084-424f-9944-68808f703673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:32.116016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f4afb-5084-424f-9944-68808f703673 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:32.176258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c440a3-04cd-4449-b981-faab450830c1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:32.212208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e37463-7510-4c19-88a8-dccb3cad0a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:32.212810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 42/83] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:32.212939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e37463-7510-4c19-88a8-dccb3cad0a5d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:32.213166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e37463-7510-4c19-88a8-dccb3cad0a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:32.213450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e37463-7510-4c19-88a8-dccb3cad0a5d None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:32.213669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f4afb-5084-424f-9944-68808f703673 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:32.214411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e37463-7510-4c19-88a8-dccb3cad0a5d None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:32.214730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e37463-7510-4c19-88a8-dccb3cad0a5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:32.216436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e37463-7510-4c19-88a8-dccb3cad0a5d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:32.227168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 42/84] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:32.228413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e37463-7510-4c19-88a8-dccb3cad0a5d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:32.242733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:32.320558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:32.320767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f4afb-5084-424f-9944-68808f703673 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:32.336944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e37463-7510-4c19-88a8-dccb3cad0a5d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:32.356021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83690f26-d1de-4992-bb9f-6a4690a77de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:32.356920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83690f26-d1de-4992-bb9f-6a4690a77de8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:32.357206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83690f26-d1de-4992-bb9f-6a4690a77de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:32.357486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83690f26-d1de-4992-bb9f-6a4690a77de8 None admin] PATH_INFO: `/v3/users/8390e06ba7334f4d8e1afbf76a9b3591` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:32.358757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83690f26-d1de-4992-bb9f-6a4690a77de8 None admin] RBAC: Authorizing `identity:get_user(user_id=8390e06ba7334f4d8e1afbf76a9b3591)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:32.360309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83690f26-d1de-4992-bb9f-6a4690a77de8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:32.362003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 43/85] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:32 2022] GET /identity/v3/users/8390e06ba7334f4d8e1afbf76a9b3591 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:32.362802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83690f26-d1de-4992-bb9f-6a4690a77de8 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:32.374062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5fa42-5397-42ec-9cb2-a1c26f268f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:32.374753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5fa42-5397-42ec-9cb2-a1c26f268f0a None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:32.375030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5fa42-5397-42ec-9cb2-a1c26f268f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:32.375308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5fa42-5397-42ec-9cb2-a1c26f268f0a None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:32.376075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5fa42-5397-42ec-9cb2-a1c26f268f0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:32.377615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5fa42-5397-42ec-9cb2-a1c26f268f0a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:32.462210 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:32.468794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83690f26-d1de-4992-bb9f-6a4690a77de8 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:32.501681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 43/86] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:14:32 2022] POST /identity/v3/users => generated 289 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:14:32.502544 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5fa42-5397-42ec-9cb2-a1c26f268f0a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:32.512365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ee546-715e-457d-9cf1-7140e587b313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:32.513587 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ee546-715e-457d-9cf1-7140e587b313 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:32.513883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ee546-715e-457d-9cf1-7140e587b313 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:32.514168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ee546-715e-457d-9cf1-7140e587b313 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:32.515738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ee546-715e-457d-9cf1-7140e587b313 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:32.517410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ee546-715e-457d-9cf1-7140e587b313 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:32.525768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 44/87] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:32 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:32.525768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ee546-715e-457d-9cf1-7140e587b313 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:32.636245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5fa42-5397-42ec-9cb2-a1c26f268f0a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:32.639001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ee546-715e-457d-9cf1-7140e587b313 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:32.676872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51584f5f-9c23-46ee-b580-52ae813c5db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:32.678304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51584f5f-9c23-46ee-b580-52ae813c5db3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:32.678617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51584f5f-9c23-46ee-b580-52ae813c5db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:32.678870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51584f5f-9c23-46ee-b580-52ae813c5db3 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:32.680132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51584f5f-9c23-46ee-b580-52ae813c5db3 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:32.680981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c1147-b0d9-4fbe-924b-fbf5dbd1f3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:32.681597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51584f5f-9c23-46ee-b580-52ae813c5db3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:32.682066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c1147-b0d9-4fbe-924b-fbf5dbd1f3da None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:32.682835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c1147-b0d9-4fbe-924b-fbf5dbd1f3da None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:32.682835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c1147-b0d9-4fbe-924b-fbf5dbd1f3da None admin] PATH_INFO: `/v3/users/8390e06ba7334f4d8e1afbf76a9b3591` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:32.683344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 44/88] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:32 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:32.684336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c1147-b0d9-4fbe-924b-fbf5dbd1f3da None admin] RBAC: Authorizing `identity:get_user(user_id=8390e06ba7334f4d8e1afbf76a9b3591)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:32.685875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c1147-b0d9-4fbe-924b-fbf5dbd1f3da None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:32.687775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 45/89] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:32 2022] GET /identity/v3/users/8390e06ba7334f4d8e1afbf76a9b3591 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:32.688652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51584f5f-9c23-46ee-b580-52ae813c5db3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:32.688765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c1147-b0d9-4fbe-924b-fbf5dbd1f3da None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:32.705987 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:32.726366 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:32.821409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c1147-b0d9-4fbe-924b-fbf5dbd1f3da None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:32.839633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51584f5f-9c23-46ee-b580-52ae813c5db3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:32.881703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ba2b9-ab66-4802-9e65-6dff2303bef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:32.882634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ba2b9-ab66-4802-9e65-6dff2303bef5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:32.882851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e5d9a-beb7-42ae-8686-0c0ffa0da69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:32.883246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ba2b9-ab66-4802-9e65-6dff2303bef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:32.883619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e5d9a-beb7-42ae-8686-0c0ffa0da69c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:32.883841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ba2b9-ab66-4802-9e65-6dff2303bef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:32.883941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e5d9a-beb7-42ae-8686-0c0ffa0da69c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:32.884179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e5d9a-beb7-42ae-8686-0c0ffa0da69c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:32.885107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ba2b9-ab66-4802-9e65-6dff2303bef5 None admin] RBAC: Adding query filter params (role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=8c495a22d3f7484cb377da68dab0bf97, user.id=8390e06ba7334f4d8e1afbf76a9b3591) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:32.885536 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ba2b9-ab66-4802-9e65-6dff2303bef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:32.887017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ba2b9-ab66-4802-9e65-6dff2303bef5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:32.896698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 46/90] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:32 2022] GET /identity/v3/role_assignments?user.id=8390e06ba7334f4d8e1afbf76a9b3591&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:32.897298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ba2b9-ab66-4802-9e65-6dff2303bef5 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:32.976070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e5d9a-beb7-42ae-8686-0c0ffa0da69c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:32.977663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e5d9a-beb7-42ae-8686-0c0ffa0da69c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:32.989424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:33.098556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 45/91] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:33.099159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e5d9a-beb7-42ae-8686-0c0ffa0da69c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:33.161677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:33.185903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ba2b9-ab66-4802-9e65-6dff2303bef5 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:33.219689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e5d9a-beb7-42ae-8686-0c0ffa0da69c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:33.246107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:33.279859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547b97c-13e6-4ce6-8f3e-77dacce95b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:33.279859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547b97c-13e6-4ce6-8f3e-77dacce95b96 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:33.279859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547b97c-13e6-4ce6-8f3e-77dacce95b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:33.279859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547b97c-13e6-4ce6-8f3e-77dacce95b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:33.294898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39f015-1d0e-4ad6-bb9c-8a5c6f825faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:33.295611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39f015-1d0e-4ad6-bb9c-8a5c6f825faa None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:33.296188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39f015-1d0e-4ad6-bb9c-8a5c6f825faa None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:33.296479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39f015-1d0e-4ad6-bb9c-8a5c6f825faa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:33.297350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39f015-1d0e-4ad6-bb9c-8a5c6f825faa None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=8c495a22d3f7484cb377da68dab0bf97, user.id=e1fa6647d77b40cd8875b5b59ec1508d) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:33.297847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39f015-1d0e-4ad6-bb9c-8a5c6f825faa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:33.300126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39f015-1d0e-4ad6-bb9c-8a5c6f825faa None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:33.308940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 47/92] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:32 2022] GET /identity/v3/role_assignments?user.id=e1fa6647d77b40cd8875b5b59ec1508d&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=03b724e03f704f2ab338b64025a3db6d => generated 259 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:33.310596 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39f015-1d0e-4ad6-bb9c-8a5c6f825faa None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:33.447973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547b97c-13e6-4ce6-8f3e-77dacce95b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:33.450528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547b97c-13e6-4ce6-8f3e-77dacce95b96 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:33.577837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39f015-1d0e-4ad6-bb9c-8a5c6f825faa None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:33.664011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 46/93] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:33.670991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869373e-1d74-4a6f-bc43-c8e498f7a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:33.671913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869373e-1d74-4a6f-bc43-c8e498f7a110 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:33.672199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869373e-1d74-4a6f-bc43-c8e498f7a110 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:33.672370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547b97c-13e6-4ce6-8f3e-77dacce95b96 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:33.672501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869373e-1d74-4a6f-bc43-c8e498f7a110 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:33.675398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869373e-1d74-4a6f-bc43-c8e498f7a110 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:33.676950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869373e-1d74-4a6f-bc43-c8e498f7a110 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:33.678849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 48/94] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:33 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:33.684827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869373e-1d74-4a6f-bc43-c8e498f7a110 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:33.705293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:33.739752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:33.943571 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869373e-1d74-4a6f-bc43-c8e498f7a110 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:33.944921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547b97c-13e6-4ce6-8f3e-77dacce95b96 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:33.984418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5014b-5f31-41a8-a0f3-80795abbfbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:33.985314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5014b-5f31-41a8-a0f3-80795abbfbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:33.985598 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5014b-5f31-41a8-a0f3-80795abbfbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:33.985875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5014b-5f31-41a8-a0f3-80795abbfbd5 None admin] PATH_INFO: `/v3/projects/382dde80ea564adbbbe1676422e118d9` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:33.987240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5014b-5f31-41a8-a0f3-80795abbfbd5 None admin] RBAC: Authorizing `identity:get_project(project_id=382dde80ea564adbbbe1676422e118d9)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:33.988910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5014b-5f31-41a8-a0f3-80795abbfbd5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:33.991417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 47/95] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:33 2022] GET /identity/v3/projects/382dde80ea564adbbbe1676422e118d9 => generated 312 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:33.997584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5014b-5f31-41a8-a0f3-80795abbfbd5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:34.091940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:34.092798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f7fff-2e16-46d6-9daa-35d513df82f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:34.093980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f7fff-2e16-46d6-9daa-35d513df82f0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:34.094476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f7fff-2e16-46d6-9daa-35d513df82f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:34.094974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f7fff-2e16-46d6-9daa-35d513df82f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:34.096576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f7fff-2e16-46d6-9daa-35d513df82f0 None admin] RBAC: Adding query filter params (role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=8c495a22d3f7484cb377da68dab0bf97, user.id=8390e06ba7334f4d8e1afbf76a9b3591) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:34.097145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f7fff-2e16-46d6-9daa-35d513df82f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:34.100773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f7fff-2e16-46d6-9daa-35d513df82f0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:34.110960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5014b-5f31-41a8-a0f3-80795abbfbd5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:34.122164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 49/96] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:33 2022] GET /identity/v3/role_assignments?user.id=8390e06ba7334f4d8e1afbf76a9b3591&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 259 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:34.151186 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ef890-258f-41d4-9f06-fb30536e8849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:34.151912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ef890-258f-41d4-9f06-fb30536e8849 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:34.152198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ef890-258f-41d4-9f06-fb30536e8849 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:34.152479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ef890-258f-41d4-9f06-fb30536e8849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:34.153341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ef890-258f-41d4-9f06-fb30536e8849 None admin] RBAC: Adding query filter params (role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=382dde80ea564adbbbe1676422e118d9, user.id=8390e06ba7334f4d8e1afbf76a9b3591) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:34.153664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ef890-258f-41d4-9f06-fb30536e8849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:34.155152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ef890-258f-41d4-9f06-fb30536e8849 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:34.161972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 48/97] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:33 2022] GET /identity/v3/role_assignments?user.id=8390e06ba7334f4d8e1afbf76a9b3591&scope.project.id=382dde80ea564adbbbe1676422e118d9&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:36.224587 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ce326-ccf2-4537-a262-0678cd8183a1 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.225199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ce326-ccf2-4537-a262-0678cd8183a1 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.225199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ce326-ccf2-4537-a262-0678cd8183a1 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.227058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 50/98] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:36.232300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50670c0d-3277-4a44-becc-fd59092216fd None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.233224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50670c0d-3277-4a44-becc-fd59092216fd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.233714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50670c0d-3277-4a44-becc-fd59092216fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.257795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-50670c0d-3277-4a44-becc-fd59092216fd None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:36.259388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-50670c0d-3277-4a44-becc-fd59092216fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:36.305905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431bf50-aa8d-4602-a831-acd74e24e89d None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.306199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431bf50-aa8d-4602-a831-acd74e24e89d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.306728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431bf50-aa8d-4602-a831-acd74e24e89d None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.308405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 51/99] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:36.314070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf0cf8-cc6e-4347-ad6c-305b3ba9f28b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.314707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf0cf8-cc6e-4347-ad6c-305b3ba9f28b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.315204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf0cf8-cc6e-4347-ad6c-305b3ba9f28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.347872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 49/100] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:36.349089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e303e3-1302-4c8c-828e-7d40dc99144b None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.349383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e303e3-1302-4c8c-828e-7d40dc99144b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.349707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e303e3-1302-4c8c-828e-7d40dc99144b None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.351268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 50/101] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:36.356759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd79ae-a457-4e9d-adbe-93ff119d61f7 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.357120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd79ae-a457-4e9d-adbe-93ff119d61f7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.357407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd79ae-a457-4e9d-adbe-93ff119d61f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.364646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4cdf0cf8-cc6e-4347-ad6c-305b3ba9f28b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:36.366744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4cdf0cf8-cc6e-4347-ad6c-305b3ba9f28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:36.377575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-28cd79ae-a457-4e9d-adbe-93ff119d61f7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:36.378805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-28cd79ae-a457-4e9d-adbe-93ff119d61f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:36.471960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:36.475072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 51/102] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:36.484201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd79ae-a457-4e9d-adbe-93ff119d61f7 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:36.522109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 52/103] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:36.530795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf0cf8-cc6e-4347-ad6c-305b3ba9f28b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:36.574322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:36.605290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd79ae-a457-4e9d-adbe-93ff119d61f7 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:36.643397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b766e7e-c548-41c6-a536-b8806dd70300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:36.644525 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b766e7e-c548-41c6-a536-b8806dd70300 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.644957 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b766e7e-c548-41c6-a536-b8806dd70300 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.645388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b766e7e-c548-41c6-a536-b8806dd70300 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.658003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b766e7e-c548-41c6-a536-b8806dd70300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3b766e7e-c548-41c6-a536-b8806dd70300 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:36.667368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:36.668432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:36.669219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:36.671721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 52/104] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:36.672776 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b766e7e-c548-41c6-a536-b8806dd70300 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:36.741293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:36.745598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf0cf8-cc6e-4347-ad6c-305b3ba9f28b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:36.787940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b766e7e-c548-41c6-a536-b8806dd70300 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:36.806204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223d18d-ad09-49f0-b998-87f9844cc442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:36.808370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223d18d-ad09-49f0-b998-87f9844cc442 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.809150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223d18d-ad09-49f0-b998-87f9844cc442 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.809970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223d18d-ad09-49f0-b998-87f9844cc442 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.826262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932886fe-854e-47af-9a7a-70b1d8287e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:36.827073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932886fe-854e-47af-9a7a-70b1d8287e92 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.827521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932886fe-854e-47af-9a7a-70b1d8287e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.827847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223d18d-ad09-49f0-b998-87f9844cc442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:36.827985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932886fe-854e-47af-9a7a-70b1d8287e92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.829259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932886fe-854e-47af-9a7a-70b1d8287e92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:36.830745 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932886fe-854e-47af-9a7a-70b1d8287e92 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:36.832453 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 53/105] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:36.833871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ccffa-5759-4544-b03b-772f23d56338 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:36.834357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ccffa-5759-4544-b03b-772f23d56338 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:36.834771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ccffa-5759-4544-b03b-772f23d56338 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:36.836239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 54/106] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:36.837223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ccffa-5759-4544-b03b-772f23d56338 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-c223d18d-ad09-49f0-b998-87f9844cc442 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:36.858896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:36.860738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:14:36.863146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:14:36.868281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 53/107] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:14:36.870131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223d18d-ad09-49f0-b998-87f9844cc442 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:36.954108 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ccffa-5759-4544-b03b-772f23d56338 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:36.984968 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:37.000152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a1092-f53e-4369-aa97-bae1761d9065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:37.000152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a1092-f53e-4369-aa97-bae1761d9065 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.000152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a1092-f53e-4369-aa97-bae1761d9065 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.000152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a1092-f53e-4369-aa97-bae1761d9065 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.000785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a1092-f53e-4369-aa97-bae1761d9065 None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:37.000785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a1092-f53e-4369-aa97-bae1761d9065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:37.000785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a1092-f53e-4369-aa97-bae1761d9065 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:37.008924 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 55/108] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:37.010382 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c241d-e938-4e16-a69f-e66a93a9228d None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.010672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c241d-e938-4e16-a69f-e66a93a9228d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.010957 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c241d-e938-4e16-a69f-e66a93a9228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.029596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-205c241d-e938-4e16-a69f-e66a93a9228d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:37.030887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-205c241d-e938-4e16-a69f-e66a93a9228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:37.044436 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:37.080258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223d18d-ad09-49f0-b998-87f9844cc442 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:37.109551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 56/109] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:37.110445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c241d-e938-4e16-a69f-e66a93a9228d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:37.189103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3d7ae-4ec3-4a7c-a25f-b46c1f5fad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:37.190802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3d7ae-4ec3-4a7c-a25f-b46c1f5fad69 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.191699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3d7ae-4ec3-4a7c-a25f-b46c1f5fad69 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.192570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3d7ae-4ec3-4a7c-a25f-b46c1f5fad69 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.194428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3d7ae-4ec3-4a7c-a25f-b46c1f5fad69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:37.205465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3d7ae-4ec3-4a7c-a25f-b46c1f5fad69 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:37.223962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c241d-e938-4e16-a69f-e66a93a9228d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:37.248525 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:37.264953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0a1e1-39cc-4eb0-8478-84ecbec172e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:37.265606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0a1e1-39cc-4eb0-8478-84ecbec172e3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.265904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0a1e1-39cc-4eb0-8478-84ecbec172e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.266394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0a1e1-39cc-4eb0-8478-84ecbec172e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.266976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0a1e1-39cc-4eb0-8478-84ecbec172e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:37.267288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0a1e1-39cc-4eb0-8478-84ecbec172e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:37.275564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0a1e1-39cc-4eb0-8478-84ecbec172e3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:37.294094 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 57/110] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:37.295567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0a1e1-39cc-4eb0-8478-84ecbec172e3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:37.377903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 54/111] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:14:36 2022] POST /identity/v3/projects => generated 302 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:14:37.384526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3d7ae-4ec3-4a7c-a25f-b46c1f5fad69 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:37.482103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0a1e1-39cc-4eb0-8478-84ecbec172e3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:37.526069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a316db-523b-4a80-b58e-ad57bb70d92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:37.526993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a316db-523b-4a80-b58e-ad57bb70d92b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.527290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a316db-523b-4a80-b58e-ad57bb70d92b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.527601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a316db-523b-4a80-b58e-ad57bb70d92b None admin] PATH_INFO: `/v3/users/8390e06ba7334f4d8e1afbf76a9b3591` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.529119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a316db-523b-4a80-b58e-ad57bb70d92b None admin] RBAC: Authorizing `identity:get_user(user_id=8390e06ba7334f4d8e1afbf76a9b3591)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:37.530623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a316db-523b-4a80-b58e-ad57bb70d92b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:37.532827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 58/112] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:37 2022] GET /identity/v3/users/8390e06ba7334f4d8e1afbf76a9b3591 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:37.544624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a316db-523b-4a80-b58e-ad57bb70d92b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:37.596661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:37.602607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3d7ae-4ec3-4a7c-a25f-b46c1f5fad69 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:37.640158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:37.646446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f2add-fd0c-4412-a23a-06e35c241b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:37.647858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f2add-fd0c-4412-a23a-06e35c241b70 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.648203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f2add-fd0c-4412-a23a-06e35c241b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.648530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f2add-fd0c-4412-a23a-06e35c241b70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.660404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a316db-523b-4a80-b58e-ad57bb70d92b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:37.661624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f2add-fd0c-4412-a23a-06e35c241b70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-7a3f2add-fd0c-4412-a23a-06e35c241b70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:37.669865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:37.670852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:14:37.672147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 55/113] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:14:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:37.672972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79d530-1600-402c-a254-b3b40e3ca61c None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.673237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79d530-1600-402c-a254-b3b40e3ca61c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.673518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79d530-1600-402c-a254-b3b40e3ca61c None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.674914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 56/114] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:37.675686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79d530-1600-402c-a254-b3b40e3ca61c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:37.712811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a42281-355e-492c-8a37-8fb30b6aef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:37.713903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a42281-355e-492c-8a37-8fb30b6aef08 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.714588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a42281-355e-492c-8a37-8fb30b6aef08 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.714996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a42281-355e-492c-8a37-8fb30b6aef08 None admin] PATH_INFO: `/v3/users/c9a693de281f41299e5061226360e7dd` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.730620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a42281-355e-492c-8a37-8fb30b6aef08 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a693de281f41299e5061226360e7dd)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:37.732471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a42281-355e-492c-8a37-8fb30b6aef08 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:37.735767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 59/115] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:37 2022] GET /identity/v3/users/c9a693de281f41299e5061226360e7dd => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:37.735767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55de7e0-ed0a-4793-bf1a-9a5d60928cdd None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.735767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55de7e0-ed0a-4793-bf1a-9a5d60928cdd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.736113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55de7e0-ed0a-4793-bf1a-9a5d60928cdd None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.742020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 60/116] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:37.742020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55de7e0-ed0a-4793-bf1a-9a5d60928cdd None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:37.808918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79d530-1600-402c-a254-b3b40e3ca61c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:37.848088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11989bf-87f5-4737-97b8-aea216ed21b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:37.848978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11989bf-87f5-4737-97b8-aea216ed21b1 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.849261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11989bf-87f5-4737-97b8-aea216ed21b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.849602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11989bf-87f5-4737-97b8-aea216ed21b1 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.850895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11989bf-87f5-4737-97b8-aea216ed21b1 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:37.852432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11989bf-87f5-4737-97b8-aea216ed21b1 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:37.854254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 57/117] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:37 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:37.855624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c179eb-6547-4f17-a9ca-c6fc2fe7e81a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.855844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c179eb-6547-4f17-a9ca-c6fc2fe7e81a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.856123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c179eb-6547-4f17-a9ca-c6fc2fe7e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.874033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55de7e0-ed0a-4793-bf1a-9a5d60928cdd None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:37.874254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-25c179eb-6547-4f17-a9ca-c6fc2fe7e81a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:37.875483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-25c179eb-6547-4f17-a9ca-c6fc2fe7e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:37.887300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:37.916486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ab57c-98bc-44dd-94f8-793308d516ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:37.917470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ab57c-98bc-44dd-94f8-793308d516ed None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.917752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ab57c-98bc-44dd-94f8-793308d516ed None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.918031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ab57c-98bc-44dd-94f8-793308d516ed None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.918800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ab57c-98bc-44dd-94f8-793308d516ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:37.919108 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ab57c-98bc-44dd-94f8-793308d516ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:37.920570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ab57c-98bc-44dd-94f8-793308d516ed None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:37.935478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 61/118] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:37.935478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ab57c-98bc-44dd-94f8-793308d516ed None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:37.956864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:37.969949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 58/119] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:37.978562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2ec1-9681-49b7-a2b7-93812b45d4c2 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:37.978562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2ec1-9681-49b7-a2b7-93812b45d4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:37.978562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2ec1-9681-49b7-a2b7-93812b45d4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:37.999917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-93ff2ec1-9681-49b7-a2b7-93812b45d4c2 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:38.002233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-93ff2ec1-9681-49b7-a2b7-93812b45d4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:38.076327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ab57c-98bc-44dd-94f8-793308d516ed None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:38.111269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 59/120] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:38.112988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff2ec1-9681-49b7-a2b7-93812b45d4c2 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:38.133463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc8949-71e3-44da-8487-5dced33214e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:38.134460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc8949-71e3-44da-8487-5dced33214e3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:38.134758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc8949-71e3-44da-8487-5dced33214e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:38.135043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc8949-71e3-44da-8487-5dced33214e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:38.182761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:38.233678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc8949-71e3-44da-8487-5dced33214e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:38.235789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc8949-71e3-44da-8487-5dced33214e3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:38.264402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff2ec1-9681-49b7-a2b7-93812b45d4c2 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:38.308845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:38.330447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab88924-4e73-40e9-9713-3715b880ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:38.331455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab88924-4e73-40e9-9713-3715b880ceed None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:38.331774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab88924-4e73-40e9-9713-3715b880ceed None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:38.332054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab88924-4e73-40e9-9713-3715b880ceed None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97/users/8390e06ba7334f4d8e1afbf76a9b3591/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:38.334429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab88924-4e73-40e9-9713-3715b880ceed None admin] RBAC: Authorizing `identity:create_grant(project_id=8c495a22d3f7484cb377da68dab0bf97, user_id=8390e06ba7334f4d8e1afbf76a9b3591, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:38.335970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab88924-4e73-40e9-9713-3715b880ceed None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:38.401902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 62/121] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:38.404614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc8949-71e3-44da-8487-5dced33214e3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:38.461221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 60/122] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:14:38 2022] PUT /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97/users/8390e06ba7334f4d8e1afbf76a9b3591/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:14:38.463887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab88924-4e73-40e9-9713-3715b880ceed None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:38.553906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:38.643858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc8949-71e3-44da-8487-5dced33214e3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:38.687057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab88924-4e73-40e9-9713-3715b880ceed None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:38.739583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914011d4-53d6-4a79-858b-45f09e942386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:38.739583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914011d4-53d6-4a79-858b-45f09e942386 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:38.739583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914011d4-53d6-4a79-858b-45f09e942386 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:38.739583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914011d4-53d6-4a79-858b-45f09e942386 None admin] PATH_INFO: `/v3/users/e1fa6647d77b40cd8875b5b59ec1508d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:38.740257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f60ac-1f1c-4576-9018-ef678786e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:38.740257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f60ac-1f1c-4576-9018-ef678786e36f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:38.740521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914011d4-53d6-4a79-858b-45f09e942386 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fa6647d77b40cd8875b5b59ec1508d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:38.740652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f60ac-1f1c-4576-9018-ef678786e36f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:38.740652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f60ac-1f1c-4576-9018-ef678786e36f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:38.745589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914011d4-53d6-4a79-858b-45f09e942386 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:38.749905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 63/123] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:38 2022] GET /identity/v3/users/e1fa6647d77b40cd8875b5b59ec1508d => generated 247 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:38.752815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914011d4-53d6-4a79-858b-45f09e942386 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:38.760963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f60ac-1f1c-4576-9018-ef678786e36f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-816f60ac-1f1c-4576-9018-ef678786e36f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:38.790328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:38.792105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:38.793563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:14:38.795843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 61/124] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:38.798012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f60ac-1f1c-4576-9018-ef678786e36f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:38.907173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914011d4-53d6-4a79-858b-45f09e942386 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:38.937082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:38.956112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbde7dc-c3c6-4b25-b457-e1ff53b8aa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:38.956960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbde7dc-c3c6-4b25-b457-e1ff53b8aa01 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:38.957246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbde7dc-c3c6-4b25-b457-e1ff53b8aa01 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:38.957638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbde7dc-c3c6-4b25-b457-e1ff53b8aa01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:38.971905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbde7dc-c3c6-4b25-b457-e1ff53b8aa01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:38.983422 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-8dbde7dc-c3c6-4b25-b457-e1ff53b8aa01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:38.985524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:38.987098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 64/125] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:14:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:38.997298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbde7dc-c3c6-4b25-b457-e1ff53b8aa01 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:39.137291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f60ac-1f1c-4576-9018-ef678786e36f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:39.180788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbde7dc-c3c6-4b25-b457-e1ff53b8aa01 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:39.217647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2a7dd-9fed-49ed-8210-6a54df584d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:39.218667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2a7dd-9fed-49ed-8210-6a54df584d21 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:39.220766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2a7dd-9fed-49ed-8210-6a54df584d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:39.220766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2a7dd-9fed-49ed-8210-6a54df584d21 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:39.221180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2a7dd-9fed-49ed-8210-6a54df584d21 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:39.223055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2a7dd-9fed-49ed-8210-6a54df584d21 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:39.226168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 65/126] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:38 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:39.226777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2a7dd-9fed-49ed-8210-6a54df584d21 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:39.228867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc7a63-68f1-4769-bc03-5f464ea73373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:39.232353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc7a63-68f1-4769-bc03-5f464ea73373 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:39.232353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc7a63-68f1-4769-bc03-5f464ea73373 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:39.232660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc7a63-68f1-4769-bc03-5f464ea73373 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:39.237271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc7a63-68f1-4769-bc03-5f464ea73373 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:39.242956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc7a63-68f1-4769-bc03-5f464ea73373 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:39.249273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 62/127] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:38 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:39.251490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc7a63-68f1-4769-bc03-5f464ea73373 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:39.328313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:39.400390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2a7dd-9fed-49ed-8210-6a54df584d21 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:39.468298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5b535-8ff3-4ae1-ab43-127aa5462563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:39.469303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5b535-8ff3-4ae1-ab43-127aa5462563 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:39.469636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5b535-8ff3-4ae1-ab43-127aa5462563 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:39.470092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5b535-8ff3-4ae1-ab43-127aa5462563 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:39.471663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5b535-8ff3-4ae1-ab43-127aa5462563 None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:39.472155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5b535-8ff3-4ae1-ab43-127aa5462563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:39.475646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5b535-8ff3-4ae1-ab43-127aa5462563 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:39.486833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc7a63-68f1-4769-bc03-5f464ea73373 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:39.499706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 66/128] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:39.501972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5b535-8ff3-4ae1-ab43-127aa5462563 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:39.506313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:39.556007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf574b-e857-41c7-b399-0940c427a64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:39.557509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf574b-e857-41c7-b399-0940c427a64a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:39.558078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf574b-e857-41c7-b399-0940c427a64a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:39.558727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf574b-e857-41c7-b399-0940c427a64a None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:39.560423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf574b-e857-41c7-b399-0940c427a64a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:39.561064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf574b-e857-41c7-b399-0940c427a64a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:39.563852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf574b-e857-41c7-b399-0940c427a64a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:39.580912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 63/129] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:14:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:39.589176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf574b-e857-41c7-b399-0940c427a64a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:39.714021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5b535-8ff3-4ae1-ab43-127aa5462563 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:39.751900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:39.760923 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637ad7-d4e1-4e6b-8bc4-01307554895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:39.762023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637ad7-d4e1-4e6b-8bc4-01307554895c None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:39.762389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637ad7-d4e1-4e6b-8bc4-01307554895c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:39.780682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637ad7-d4e1-4e6b-8bc4-01307554895c None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97/users/e1fa6647d77b40cd8875b5b59ec1508d/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:39.780682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3637ad7-d4e1-4e6b-8bc4-01307554895c None admin] RBAC: Authorizing `identity:create_grant(project_id=8c495a22d3f7484cb377da68dab0bf97, user_id=e1fa6647d77b40cd8875b5b59ec1508d, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:39.780682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3637ad7-d4e1-4e6b-8bc4-01307554895c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:39.796888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf574b-e857-41c7-b399-0940c427a64a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:39.835241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a4f3e-a1af-445f-b5d7-24550af61a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:39.835955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a4f3e-a1af-445f-b5d7-24550af61a35 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:39.836244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a4f3e-a1af-445f-b5d7-24550af61a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:39.836526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a4f3e-a1af-445f-b5d7-24550af61a35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:39.837400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a4f3e-a1af-445f-b5d7-24550af61a35 None admin] RBAC: Adding query filter params (role.id=536229a47a9d4a03b6d0b1c59fd8f39a, scope.project.id=8c495a22d3f7484cb377da68dab0bf97, user.id=c9a693de281f41299e5061226360e7dd) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:39.837716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a4f3e-a1af-445f-b5d7-24550af61a35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:39.839209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a4f3e-a1af-445f-b5d7-24550af61a35 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:39.845684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 64/130] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:39 2022] GET /identity/v3/role_assignments?user.id=c9a693de281f41299e5061226360e7dd&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=536229a47a9d4a03b6d0b1c59fd8f39a => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:39.846566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a4f3e-a1af-445f-b5d7-24550af61a35 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:39.892802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:39.987853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 67/131] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:14:39 2022] PUT /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97/users/e1fa6647d77b40cd8875b5b59ec1508d/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:14:39.996893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637ad7-d4e1-4e6b-8bc4-01307554895c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:40.001616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a4f3e-a1af-445f-b5d7-24550af61a35 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:40.109093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dae940-d801-4a5e-91e5-23e66120ff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:40.112243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dae940-d801-4a5e-91e5-23e66120ff0e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:40.113144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dae940-d801-4a5e-91e5-23e66120ff0e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:40.113849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dae940-d801-4a5e-91e5-23e66120ff0e None admin] PATH_INFO: `/v3/users/8390e06ba7334f4d8e1afbf76a9b3591` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:40.118079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dae940-d801-4a5e-91e5-23e66120ff0e None admin] RBAC: Authorizing `identity:get_user(user_id=8390e06ba7334f4d8e1afbf76a9b3591)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:40.121865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dae940-d801-4a5e-91e5-23e66120ff0e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:40.126226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 65/132] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:39 2022] GET /identity/v3/users/8390e06ba7334f4d8e1afbf76a9b3591 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:40.129881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95c0b6-b77b-4442-8d6a-b386c1c45589 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:40.131495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95c0b6-b77b-4442-8d6a-b386c1c45589 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:40.132333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95c0b6-b77b-4442-8d6a-b386c1c45589 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:40.135941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 66/133] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:40.138178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95c0b6-b77b-4442-8d6a-b386c1c45589 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:40.226830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637ad7-d4e1-4e6b-8bc4-01307554895c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:40.277488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca52c01-063d-49a3-a5f1-c1f7069f9403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:40.277858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca52c01-063d-49a3-a5f1-c1f7069f9403 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:40.278276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca52c01-063d-49a3-a5f1-c1f7069f9403 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:40.278666 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca52c01-063d-49a3-a5f1-c1f7069f9403 None admin] PATH_INFO: `/v3/users/8390e06ba7334f4d8e1afbf76a9b3591` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:40.280240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca52c01-063d-49a3-a5f1-c1f7069f9403 None admin] RBAC: Authorizing `identity:get_user(user_id=8390e06ba7334f4d8e1afbf76a9b3591)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:40.281898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca52c01-063d-49a3-a5f1-c1f7069f9403 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:40.288209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 68/134] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:39 2022] GET /identity/v3/users/8390e06ba7334f4d8e1afbf76a9b3591 => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:40.288209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ed65b-683a-4dd4-9b59-2cd1d55c6d89 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:40.288209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ed65b-683a-4dd4-9b59-2cd1d55c6d89 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:40.288209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ed65b-683a-4dd4-9b59-2cd1d55c6d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:40.309023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d23ed65b-683a-4dd4-9b59-2cd1d55c6d89 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:40.311119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d23ed65b-683a-4dd4-9b59-2cd1d55c6d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:40.438419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95c0b6-b77b-4442-8d6a-b386c1c45589 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:40.444339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:40.464502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:40.503590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 69/135] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:40.506214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ed65b-683a-4dd4-9b59-2cd1d55c6d89 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:40.563983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acb560-7d69-4c02-bb47-17ebdf62c207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:40.565613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acb560-7d69-4c02-bb47-17ebdf62c207 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:40.566124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acb560-7d69-4c02-bb47-17ebdf62c207 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:40.566631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acb560-7d69-4c02-bb47-17ebdf62c207 None admin] PATH_INFO: `/v3/projects/382dde80ea564adbbbe1676422e118d9` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:40.573045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acb560-7d69-4c02-bb47-17ebdf62c207 None admin] RBAC: Authorizing `identity:get_project(project_id=382dde80ea564adbbbe1676422e118d9)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:40.576104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acb560-7d69-4c02-bb47-17ebdf62c207 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:40.579479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 67/136] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:40 2022] GET /identity/v3/projects/382dde80ea564adbbbe1676422e118d9 => generated 312 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:40.585950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acb560-7d69-4c02-bb47-17ebdf62c207 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:40.778571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ed65b-683a-4dd4-9b59-2cd1d55c6d89 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:40.826933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acb560-7d69-4c02-bb47-17ebdf62c207 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:40.893688 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb66119-d13b-4e5e-86ab-0ed4bad2f216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:40.895262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb66119-d13b-4e5e-86ab-0ed4bad2f216 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:40.895791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb66119-d13b-4e5e-86ab-0ed4bad2f216 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:40.896823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb66119-d13b-4e5e-86ab-0ed4bad2f216 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:40.902354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64c421-4beb-49da-96b9-e6af4b6727d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:40.903791 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64c421-4beb-49da-96b9-e6af4b6727d5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:40.904267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64c421-4beb-49da-96b9-e6af4b6727d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:40.904755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64c421-4beb-49da-96b9-e6af4b6727d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:40.907281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb66119-d13b-4e5e-86ab-0ed4bad2f216 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:40.910105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb66119-d13b-4e5e-86ab-0ed4bad2f216 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:40.914059 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 70/137] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:40 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:40.915829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff64c421-4beb-49da-96b9-e6af4b6727d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:40.918557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff64c421-4beb-49da-96b9-e6af4b6727d5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:40.921827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 68/138] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:40.925096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64c421-4beb-49da-96b9-e6af4b6727d5 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:40.925742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb66119-d13b-4e5e-86ab-0ed4bad2f216 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:41.046389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:41.066245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:41.188084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb66119-d13b-4e5e-86ab-0ed4bad2f216 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:41.197699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64c421-4beb-49da-96b9-e6af4b6727d5 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:41.267985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cfc7f-f69c-4ce1-8b32-9082b2fef1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:41.272752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cfc7f-f69c-4ce1-8b32-9082b2fef1fc None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:41.272752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cfc7f-f69c-4ce1-8b32-9082b2fef1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:41.272752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cfc7f-f69c-4ce1-8b32-9082b2fef1fc None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97/users/8390e06ba7334f4d8e1afbf76a9b3591/roles/296546e6917a4eb6b58cac525a76f8a6` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:41.276986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cfc7f-f69c-4ce1-8b32-9082b2fef1fc None admin] RBAC: Authorizing `identity:create_grant(project_id=8c495a22d3f7484cb377da68dab0bf97, user_id=8390e06ba7334f4d8e1afbf76a9b3591, role_id=296546e6917a4eb6b58cac525a76f8a6)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:41.284275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cfc7f-f69c-4ce1-8b32-9082b2fef1fc None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:41.293240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04002d8-3c6c-42ae-b968-1a5657c97647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:41.294184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04002d8-3c6c-42ae-b968-1a5657c97647 None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:41.294471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04002d8-3c6c-42ae-b968-1a5657c97647 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:41.294813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04002d8-3c6c-42ae-b968-1a5657c97647 None admin] PATH_INFO: `/v3/projects/382dde80ea564adbbbe1676422e118d9/users/8390e06ba7334f4d8e1afbf76a9b3591/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:41.299723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04002d8-3c6c-42ae-b968-1a5657c97647 None admin] RBAC: Authorizing `identity:create_grant(project_id=382dde80ea564adbbbe1676422e118d9, user_id=8390e06ba7334f4d8e1afbf76a9b3591, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:41.299723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04002d8-3c6c-42ae-b968-1a5657c97647 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:41.380060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 69/139] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:14:40 2022] PUT /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97/users/8390e06ba7334f4d8e1afbf76a9b3591/roles/296546e6917a4eb6b58cac525a76f8a6 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:14:41.382783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cfc7f-f69c-4ce1-8b32-9082b2fef1fc None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:41.409231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 71/140] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:14:40 2022] PUT /identity/v3/projects/382dde80ea564adbbbe1676422e118d9/users/8390e06ba7334f4d8e1afbf76a9b3591/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:14:41.597250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:41.607147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cfc7f-f69c-4ce1-8b32-9082b2fef1fc None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:41.682118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8318073-0c3b-4b98-b240-a90348e9e6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:41.683875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8318073-0c3b-4b98-b240-a90348e9e6d4 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:41.684681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8318073-0c3b-4b98-b240-a90348e9e6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:41.685469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8318073-0c3b-4b98-b240-a90348e9e6d4 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:41.687212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8318073-0c3b-4b98-b240-a90348e9e6d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:41.690586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8318073-0c3b-4b98-b240-a90348e9e6d4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:41.856716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 70/141] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:14:41 2022] POST /identity/v3/users => generated 283 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:14:42.167968 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565342-8408-4592-b53b-301ed16c3de2 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:42.168449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565342-8408-4592-b53b-301ed16c3de2 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:42.168449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565342-8408-4592-b53b-301ed16c3de2 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:42.169899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 72/142] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:42.182246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec63327-74a6-4120-a347-278d0450f68e None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:42.182894 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec63327-74a6-4120-a347-278d0450f68e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:42.183373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec63327-74a6-4120-a347-278d0450f68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:42.202507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-fec63327-74a6-4120-a347-278d0450f68e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:42.208980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fec63327-74a6-4120-a347-278d0450f68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:42.305244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 71/143] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:42.340881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565342-8408-4592-b53b-301ed16c3de2 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:42.488245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565342-8408-4592-b53b-301ed16c3de2 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:42.504875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:42.539571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686dc71-32df-4481-b9fb-107d778ef77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:42.540470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686dc71-32df-4481-b9fb-107d778ef77a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:42.540750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686dc71-32df-4481-b9fb-107d778ef77a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:42.541025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686dc71-32df-4481-b9fb-107d778ef77a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:42.554169 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686dc71-32df-4481-b9fb-107d778ef77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-9686dc71-32df-4481-b9fb-107d778ef77a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:42.571278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:42.572262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:42.573019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 73/144] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:42.580746 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec63327-74a6-4120-a347-278d0450f68e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:42.647252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24104ba3-c510-41d1-9c45-d3bc899993a6 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:42.647557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24104ba3-c510-41d1-9c45-d3bc899993a6 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:42.648139 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24104ba3-c510-41d1-9c45-d3bc899993a6 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:42.650175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 74/145] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:42.660802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518750d-14da-4987-8560-11b9d3c4ce45 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:42.661440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518750d-14da-4987-8560-11b9d3c4ce45 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:42.662708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518750d-14da-4987-8560-11b9d3c4ce45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:42.700424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a518750d-14da-4987-8560-11b9d3c4ce45 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:42.701665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a518750d-14da-4987-8560-11b9d3c4ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:42.741151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:42.781658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec63327-74a6-4120-a347-278d0450f68e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:42.823696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 75/146] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:42.825435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c12e2-cb4d-4fae-a54e-4a4c3e55e10e None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:42.825883 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c12e2-cb4d-4fae-a54e-4a4c3e55e10e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:42.826302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c12e2-cb4d-4fae-a54e-4a4c3e55e10e None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:42.827608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bb32d-1fd2-4367-bf76-95a2f641ab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:42.827907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 76/147] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:42.828889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bb32d-1fd2-4367-bf76-95a2f641ab5a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:42.829430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bb32d-1fd2-4367-bf76-95a2f641ab5a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:42.829837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bb32d-1fd2-4367-bf76-95a2f641ab5a None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:42.830876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bb32d-1fd2-4367-bf76-95a2f641ab5a None admin] RBAC: Adding query filter params (name=member) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:42.831336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bb32d-1fd2-4367-bf76-95a2f641ab5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:42.833607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bb32d-1fd2-4367-bf76-95a2f641ab5a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:42.834531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a6443-8f2c-40e2-84c7-7bf7f76b874e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:42.834888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a6443-8f2c-40e2-84c7-7bf7f76b874e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:42.835233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a6443-8f2c-40e2-84c7-7bf7f76b874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:42.848993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 72/148] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:42.854514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bb32d-1fd2-4367-bf76-95a2f641ab5a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:42.858440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-b38a6443-8f2c-40e2-84c7-7bf7f76b874e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:42.860965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b38a6443-8f2c-40e2-84c7-7bf7f76b874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:42.978454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 77/149] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:42.979798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a6443-8f2c-40e2-84c7-7bf7f76b874e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:43.036688 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:43.086414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bb32d-1fd2-4367-bf76-95a2f641ab5a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:43.204165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef2917-892f-47d4-9c09-17bd76227f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:43.206319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef2917-892f-47d4-9c09-17bd76227f2e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:43.207639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef2917-892f-47d4-9c09-17bd76227f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:43.208580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef2917-892f-47d4-9c09-17bd76227f2e None admin] PATH_INFO: `/v3/users/8390e06ba7334f4d8e1afbf76a9b3591` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:43.212189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a6443-8f2c-40e2-84c7-7bf7f76b874e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:43.214337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef2917-892f-47d4-9c09-17bd76227f2e None admin] RBAC: Authorizing `identity:get_user(user_id=8390e06ba7334f4d8e1afbf76a9b3591)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:43.219125 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef2917-892f-47d4-9c09-17bd76227f2e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:43.222806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 73/150] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:42 2022] GET /identity/v3/users/8390e06ba7334f4d8e1afbf76a9b3591 => generated 275 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:43.225536 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef2917-892f-47d4-9c09-17bd76227f2e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:43.276595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:43.294857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844ea71-7558-4452-a536-55f7b9c2b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:43.294857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844ea71-7558-4452-a536-55f7b9c2b710 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:43.294857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844ea71-7558-4452-a536-55f7b9c2b710 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:43.294857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844ea71-7558-4452-a536-55f7b9c2b710 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:43.311887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844ea71-7558-4452-a536-55f7b9c2b710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-6844ea71-7558-4452-a536-55f7b9c2b710 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:43.323273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:43.324530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:43.325483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:43.330061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 78/151] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:14:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:43.331946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844ea71-7558-4452-a536-55f7b9c2b710 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:43.424090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef2917-892f-47d4-9c09-17bd76227f2e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:43.493418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f16ce1-bd4c-4d30-a5c2-1ea6415254ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:43.494487 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f16ce1-bd4c-4d30-a5c2-1ea6415254ae None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:43.495190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f16ce1-bd4c-4d30-a5c2-1ea6415254ae None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:43.496496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f16ce1-bd4c-4d30-a5c2-1ea6415254ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:43.521855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f16ce1-bd4c-4d30-a5c2-1ea6415254ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:43.529950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-15f16ce1-bd4c-4d30-a5c2-1ea6415254ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:43.536677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:43.538289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:14:43.539450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:14:43.542128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 74/152] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:43.547176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f16ce1-bd4c-4d30-a5c2-1ea6415254ae None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:43.560901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844ea71-7558-4452-a536-55f7b9c2b710 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:43.628463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c70cd5-a520-44cf-b543-4b8413c69ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:43.629301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c70cd5-a520-44cf-b543-4b8413c69ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:43.629711 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c70cd5-a520-44cf-b543-4b8413c69ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:43.630101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c70cd5-a520-44cf-b543-4b8413c69ba3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:43.733055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:43.763714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c70cd5-a520-44cf-b543-4b8413c69ba3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:43.765250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c70cd5-a520-44cf-b543-4b8413c69ba3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:43.788309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f16ce1-bd4c-4d30-a5c2-1ea6415254ae None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:43.884422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f6fa6-7c30-484c-a26e-306b79a13c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:43.885032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6fa6-7c30-484c-a26e-306b79a13c6b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:43.885319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6fa6-7c30-484c-a26e-306b79a13c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:43.885609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6fa6-7c30-484c-a26e-306b79a13c6b None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:43.886507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6fa6-7c30-484c-a26e-306b79a13c6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:43.886824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6fa6-7c30-484c-a26e-306b79a13c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:43.888308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6fa6-7c30-484c-a26e-306b79a13c6b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:43.904529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 75/153] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:43.905141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f6fa6-7c30-484c-a26e-306b79a13c6b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:44.035956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:44.047035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 79/154] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:44.051558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c70cd5-a520-44cf-b543-4b8413c69ba3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:44.179759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f6fa6-7c30-484c-a26e-306b79a13c6b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:44.281956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666f67a-96d5-4db5-a14f-65afba7a232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:44.283152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666f67a-96d5-4db5-a14f-65afba7a232c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:44.283722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666f67a-96d5-4db5-a14f-65afba7a232c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:44.288182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666f67a-96d5-4db5-a14f-65afba7a232c None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:44.289850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666f67a-96d5-4db5-a14f-65afba7a232c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:44.290425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666f67a-96d5-4db5-a14f-65afba7a232c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:44.297296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666f67a-96d5-4db5-a14f-65afba7a232c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:44.306491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:44.325172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 76/155] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:44.326718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666f67a-96d5-4db5-a14f-65afba7a232c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:44.346050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c70cd5-a520-44cf-b543-4b8413c69ba3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:44.418663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0801928-c562-4430-967e-1eb299405ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:44.419611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0801928-c562-4430-967e-1eb299405ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:44.419895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0801928-c562-4430-967e-1eb299405ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:44.420171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0801928-c562-4430-967e-1eb299405ca2 None admin] PATH_INFO: `/v3/users/e1fa6647d77b40cd8875b5b59ec1508d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:44.423827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0801928-c562-4430-967e-1eb299405ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fa6647d77b40cd8875b5b59ec1508d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:44.425456 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0801928-c562-4430-967e-1eb299405ca2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:44.427289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 80/156] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:44 2022] GET /identity/v3/users/e1fa6647d77b40cd8875b5b59ec1508d => generated 247 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:44.428737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0801928-c562-4430-967e-1eb299405ca2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:44.558777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666f67a-96d5-4db5-a14f-65afba7a232c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:44.592544 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:44.638669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de1a1d-1409-4e8e-87c8-ceb0f1a3091c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:44.640889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de1a1d-1409-4e8e-87c8-ceb0f1a3091c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:44.641558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de1a1d-1409-4e8e-87c8-ceb0f1a3091c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:44.642185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de1a1d-1409-4e8e-87c8-ceb0f1a3091c None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:44.644979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de1a1d-1409-4e8e-87c8-ceb0f1a3091c None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:44.649831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de1a1d-1409-4e8e-87c8-ceb0f1a3091c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:44.654060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 77/157] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:44 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:44.656212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de1a1d-1409-4e8e-87c8-ceb0f1a3091c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:44.688731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0801928-c562-4430-967e-1eb299405ca2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:44.797612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745441b1-b48e-4eb4-a1d4-0143c97fa78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:44.798863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745441b1-b48e-4eb4-a1d4-0143c97fa78d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:44.799296 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745441b1-b48e-4eb4-a1d4-0143c97fa78d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:44.799775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745441b1-b48e-4eb4-a1d4-0143c97fa78d None admin] PATH_INFO: `/v3/users/c9a693de281f41299e5061226360e7dd` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:44.804608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745441b1-b48e-4eb4-a1d4-0143c97fa78d None admin] RBAC: Authorizing `identity:get_user(user_id=c9a693de281f41299e5061226360e7dd)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:44.806469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745441b1-b48e-4eb4-a1d4-0143c97fa78d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:44.808911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 81/158] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:44 2022] GET /identity/v3/users/c9a693de281f41299e5061226360e7dd => generated 289 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:44.809998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745441b1-b48e-4eb4-a1d4-0143c97fa78d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:44.817054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:44.896868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de1a1d-1409-4e8e-87c8-ceb0f1a3091c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:45.009353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b598b6-b5b8-4212-80ec-5b28e45d9ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:45.010435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b598b6-b5b8-4212-80ec-5b28e45d9ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:45.010922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b598b6-b5b8-4212-80ec-5b28e45d9ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:45.011395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b598b6-b5b8-4212-80ec-5b28e45d9ea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:45.014811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745441b1-b48e-4eb4-a1d4-0143c97fa78d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:45.068503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:45.078916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd508c83-2345-4e0a-a2a5-f0eba78f5368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:45.079628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd508c83-2345-4e0a-a2a5-f0eba78f5368 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:45.082288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd508c83-2345-4e0a-a2a5-f0eba78f5368 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:45.082288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd508c83-2345-4e0a-a2a5-f0eba78f5368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:45.082288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd508c83-2345-4e0a-a2a5-f0eba78f5368 None admin] RBAC: Adding query filter params (role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=8c495a22d3f7484cb377da68dab0bf97, user.id=8390e06ba7334f4d8e1afbf76a9b3591) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:45.082288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd508c83-2345-4e0a-a2a5-f0eba78f5368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:45.083114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd508c83-2345-4e0a-a2a5-f0eba78f5368 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:45.095713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 82/159] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:44 2022] GET /identity/v3/role_assignments?user.id=8390e06ba7334f4d8e1afbf76a9b3591&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:45.096617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd508c83-2345-4e0a-a2a5-f0eba78f5368 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:45.234951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b598b6-b5b8-4212-80ec-5b28e45d9ea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:45.238259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b598b6-b5b8-4212-80ec-5b28e45d9ea9 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:45.288366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd508c83-2345-4e0a-a2a5-f0eba78f5368 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:45.379421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5a858-4a7f-4465-9136-a086696eb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:45.384851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5a858-4a7f-4465-9136-a086696eb3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:45.385149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5a858-4a7f-4465-9136-a086696eb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:45.385429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5a858-4a7f-4465-9136-a086696eb3a4 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:45.387552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5a858-4a7f-4465-9136-a086696eb3a4 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:45.389422 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5a858-4a7f-4465-9136-a086696eb3a4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:45.392580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 83/160] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:45 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:45.393930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf7c45-8626-4f9d-a5b4-1d2482ef9d43 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:45.394217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf7c45-8626-4f9d-a5b4-1d2482ef9d43 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:45.394561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf7c45-8626-4f9d-a5b4-1d2482ef9d43 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:45.397793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 84/161] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:45.399067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf7c45-8626-4f9d-a5b4-1d2482ef9d43 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:45.468052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:45.537760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 78/162] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 883 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:45.539514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc53d1-a708-45fa-98ec-473a12308572 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:45.540118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc53d1-a708-45fa-98ec-473a12308572 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:45.540615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc53d1-a708-45fa-98ec-473a12308572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:45.580723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-fabc53d1-a708-45fa-98ec-473a12308572 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:45.582378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fabc53d1-a708-45fa-98ec-473a12308572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:45.597396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:45.656074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf7c45-8626-4f9d-a5b4-1d2482ef9d43 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:45.703465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95721847-4aee-47c6-94cf-c2ac701b7392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:45.704428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95721847-4aee-47c6-94cf-c2ac701b7392 None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:45.704712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95721847-4aee-47c6-94cf-c2ac701b7392 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:45.704990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95721847-4aee-47c6-94cf-c2ac701b7392 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97/users/c9a693de281f41299e5061226360e7dd/roles/536229a47a9d4a03b6d0b1c59fd8f39a` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:45.707281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95721847-4aee-47c6-94cf-c2ac701b7392 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c495a22d3f7484cb377da68dab0bf97, user_id=c9a693de281f41299e5061226360e7dd, role_id=536229a47a9d4a03b6d0b1c59fd8f39a)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:45.709838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95721847-4aee-47c6-94cf-c2ac701b7392 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:45.714805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 79/163] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:45.716007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc53d1-a708-45fa-98ec-473a12308572 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:45.832069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 85/164] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:14:45 2022] PUT /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97/users/c9a693de281f41299e5061226360e7dd/roles/536229a47a9d4a03b6d0b1c59fd8f39a => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:14:45.832773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95721847-4aee-47c6-94cf-c2ac701b7392 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:45.853504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:45.863007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc53d1-a708-45fa-98ec-473a12308572 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:45.956936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2c35a-3c13-4a6c-b855-03aa89a18a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:45.957746 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c35a-3c13-4a6c-b855-03aa89a18a6f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:45.958028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c35a-3c13-4a6c-b855-03aa89a18a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:45.958300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c35a-3c13-4a6c-b855-03aa89a18a6f None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:45.960513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2c35a-3c13-4a6c-b855-03aa89a18a6f None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:45.962242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2c35a-3c13-4a6c-b855-03aa89a18a6f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:45.964310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 80/165] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:45 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:45.965811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05701de8-3fc4-42ca-b4da-f0acf3bd980d None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:45.966280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05701de8-3fc4-42ca-b4da-f0acf3bd980d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:45.966696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05701de8-3fc4-42ca-b4da-f0acf3bd980d None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:45.968563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 81/166] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:45.974948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05701de8-3fc4-42ca-b4da-f0acf3bd980d None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:46.075569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95721847-4aee-47c6-94cf-c2ac701b7392 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:46.119515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:46.132854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a25b3-63e0-40dd-b39c-49ae17767926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:46.133992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a25b3-63e0-40dd-b39c-49ae17767926 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:46.134426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a25b3-63e0-40dd-b39c-49ae17767926 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:46.134841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a25b3-63e0-40dd-b39c-49ae17767926 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:46.149908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a25b3-63e0-40dd-b39c-49ae17767926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:46.156075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05701de8-3fc4-42ca-b4da-f0acf3bd980d None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-597a25b3-63e0-40dd-b39c-49ae17767926 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:46.159853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:46.160917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:46.162244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:46.166293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 86/167] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:14:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:46.167996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca891c-24ac-4106-9700-c85a943daef3 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:46.168460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca891c-24ac-4106-9700-c85a943daef3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:46.169093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca891c-24ac-4106-9700-c85a943daef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:46.194020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8bca891c-24ac-4106-9700-c85a943daef3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:46.196487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8bca891c-24ac-4106-9700-c85a943daef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:46.230024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95fa68-c14a-41aa-9aed-34cc6e0b596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:46.231207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95fa68-c14a-41aa-9aed-34cc6e0b596e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:46.231657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95fa68-c14a-41aa-9aed-34cc6e0b596e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:46.232124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95fa68-c14a-41aa-9aed-34cc6e0b596e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:46.233757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95fa68-c14a-41aa-9aed-34cc6e0b596e None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=8c495a22d3f7484cb377da68dab0bf97, user.id=e1fa6647d77b40cd8875b5b59ec1508d) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:46.234284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95fa68-c14a-41aa-9aed-34cc6e0b596e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:46.239600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95fa68-c14a-41aa-9aed-34cc6e0b596e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:46.252891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 82/168] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:45 2022] GET /identity/v3/role_assignments?user.id=e1fa6647d77b40cd8875b5b59ec1508d&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=03b724e03f704f2ab338b64025a3db6d => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:46.252891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95fa68-c14a-41aa-9aed-34cc6e0b596e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:46.359644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:46.397442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 87/169] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:46.443770 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca891c-24ac-4106-9700-c85a943daef3 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:46.531653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95fa68-c14a-41aa-9aed-34cc6e0b596e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:46.594416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734e65b-712c-4a59-b08a-ae253e7324cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:46.595508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734e65b-712c-4a59-b08a-ae253e7324cb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:46.596009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734e65b-712c-4a59-b08a-ae253e7324cb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:46.596468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734e65b-712c-4a59-b08a-ae253e7324cb None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:46.597744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734e65b-712c-4a59-b08a-ae253e7324cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:46.598220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734e65b-712c-4a59-b08a-ae253e7324cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:46.600712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734e65b-712c-4a59-b08a-ae253e7324cb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:46.616025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 83/170] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:14:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:46.625357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734e65b-712c-4a59-b08a-ae253e7324cb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:46.773389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca891c-24ac-4106-9700-c85a943daef3 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:46.804257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:46.855844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf586ad-3f62-4daf-aeaf-92c6dc38d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:46.857044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf586ad-3f62-4daf-aeaf-92c6dc38d31f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:46.857462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf586ad-3f62-4daf-aeaf-92c6dc38d31f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:46.857881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf586ad-3f62-4daf-aeaf-92c6dc38d31f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:46.895572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf586ad-3f62-4daf-aeaf-92c6dc38d31f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-0bf586ad-3f62-4daf-aeaf-92c6dc38d31f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:46.910112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:46.911937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:46.913477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 88/171] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:46.919573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf586ad-3f62-4daf-aeaf-92c6dc38d31f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:46.974952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:47.002010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734e65b-712c-4a59-b08a-ae253e7324cb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:47.076745 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497396-4745-47d5-b2e5-13ce3b6e4a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:47.078281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497396-4745-47d5-b2e5-13ce3b6e4a50 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:47.078804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497396-4745-47d5-b2e5-13ce3b6e4a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:47.079388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497396-4745-47d5-b2e5-13ce3b6e4a50 None admin] PATH_INFO: `/v3/users/8390e06ba7334f4d8e1afbf76a9b3591` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:47.082448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22497396-4745-47d5-b2e5-13ce3b6e4a50 None admin] RBAC: Authorizing `identity:get_user(user_id=8390e06ba7334f4d8e1afbf76a9b3591)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:47.088064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22497396-4745-47d5-b2e5-13ce3b6e4a50 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:47.092685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 84/172] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:46 2022] GET /identity/v3/users/8390e06ba7334f4d8e1afbf76a9b3591 => generated 275 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:47.107241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497396-4745-47d5-b2e5-13ce3b6e4a50 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:47.206631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf586ad-3f62-4daf-aeaf-92c6dc38d31f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:47.266951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f92266-3ddc-4042-a165-9e4b42dab6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:47.271905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f92266-3ddc-4042-a165-9e4b42dab6ed None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:47.271905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f92266-3ddc-4042-a165-9e4b42dab6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:47.271905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f92266-3ddc-4042-a165-9e4b42dab6ed None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:47.271905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f92266-3ddc-4042-a165-9e4b42dab6ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:47.271905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f92266-3ddc-4042-a165-9e4b42dab6ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:47.272608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f92266-3ddc-4042-a165-9e4b42dab6ed None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:47.290349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 89/173] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:47.303518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f92266-3ddc-4042-a165-9e4b42dab6ed None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:47.360506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:47.474915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497396-4745-47d5-b2e5-13ce3b6e4a50 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:47.508581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f92266-3ddc-4042-a165-9e4b42dab6ed None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:47.560733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22786ec3-3c1b-40c0-9064-18a552b78b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:47.561753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22786ec3-3c1b-40c0-9064-18a552b78b1a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:47.562231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22786ec3-3c1b-40c0-9064-18a552b78b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:47.562710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22786ec3-3c1b-40c0-9064-18a552b78b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:47.640075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913b90-b883-47b5-ba25-d7f3c189b2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:47.641792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913b90-b883-47b5-ba25-d7f3c189b2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:47.642278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913b90-b883-47b5-ba25-d7f3c189b2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:47.642830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913b90-b883-47b5-ba25-d7f3c189b2e1 None admin] PATH_INFO: `/v3/users/8390e06ba7334f4d8e1afbf76a9b3591` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:47.650485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22913b90-b883-47b5-ba25-d7f3c189b2e1 None admin] RBAC: Authorizing `identity:get_user(user_id=8390e06ba7334f4d8e1afbf76a9b3591)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:47.652880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22913b90-b883-47b5-ba25-d7f3c189b2e1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:47.664006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 90/174] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:47 2022] GET /identity/v3/users/8390e06ba7334f4d8e1afbf76a9b3591 => generated 275 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:47.670859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913b90-b883-47b5-ba25-d7f3c189b2e1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:47.700286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:47.853961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22786ec3-3c1b-40c0-9064-18a552b78b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:47.859142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22786ec3-3c1b-40c0-9064-18a552b78b1a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:48.007835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913b90-b883-47b5-ba25-d7f3c189b2e1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:48.011881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:48.084157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78102eea-bd75-4d64-8256-edeb97bcb5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:48.085824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78102eea-bd75-4d64-8256-edeb97bcb5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.086612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78102eea-bd75-4d64-8256-edeb97bcb5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.087372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78102eea-bd75-4d64-8256-edeb97bcb5e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.114579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 85/175] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1010 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:48.115815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59024f-5991-40ee-b775-f98b7d9c3e4f None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.116112 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59024f-5991-40ee-b775-f98b7d9c3e4f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.116396 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59024f-5991-40ee-b775-f98b7d9c3e4f None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.118271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 86/176] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:48.119278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87490826-87b4-47c9-b0da-ac3aa9617b87 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.126997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87490826-87b4-47c9-b0da-ac3aa9617b87 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.126997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87490826-87b4-47c9-b0da-ac3aa9617b87 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.126997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 87/177] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:48.126997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87490826-87b4-47c9-b0da-ac3aa9617b87 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:48.228049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:48.229884 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78102eea-bd75-4d64-8256-edeb97bcb5e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:48.231888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78102eea-bd75-4d64-8256-edeb97bcb5e4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:48.282615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87490826-87b4-47c9-b0da-ac3aa9617b87 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:48.329345 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be98a6a-c4b8-4982-840b-87e0476c7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:48.330484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be98a6a-c4b8-4982-840b-87e0476c7358 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.330605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be98a6a-c4b8-4982-840b-87e0476c7358 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.330817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be98a6a-c4b8-4982-840b-87e0476c7358 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.332422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be98a6a-c4b8-4982-840b-87e0476c7358 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:48.334167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be98a6a-c4b8-4982-840b-87e0476c7358 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:48.335958 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 88/178] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:48 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:48.337280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce3fe0-8f03-41f9-808e-a8e0580a35ae None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.337564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce3fe0-8f03-41f9-808e-a8e0580a35ae None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.337837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce3fe0-8f03-41f9-808e-a8e0580a35ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.361050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a0ce3fe0-8f03-41f9-808e-a8e0580a35ae None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:48.362629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a0ce3fe0-8f03-41f9-808e-a8e0580a35ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:48.367553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 91/179] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:48.373925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93fa85-7a71-4fa5-90cb-27b160496f12 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.374966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93fa85-7a71-4fa5-90cb-27b160496f12 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.375460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93fa85-7a71-4fa5-90cb-27b160496f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.382384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:48.397211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7f93fa85-7a71-4fa5-90cb-27b160496f12 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:48.399069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7f93fa85-7a71-4fa5-90cb-27b160496f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:48.445713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 89/180] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:48.446263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce3fe0-8f03-41f9-808e-a8e0580a35ae None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:48.489210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 92/181] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:48.491135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93fa85-7a71-4fa5-90cb-27b160496f12 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:48.539721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:48.600146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce3fe0-8f03-41f9-808e-a8e0580a35ae None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:48.641517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93fa85-7a71-4fa5-90cb-27b160496f12 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:48.648410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59846b-41c9-4182-8cf6-1f14146ee1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:48.649718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59846b-41c9-4182-8cf6-1f14146ee1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.650325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59846b-41c9-4182-8cf6-1f14146ee1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.651568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59846b-41c9-4182-8cf6-1f14146ee1d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.652739 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59846b-41c9-4182-8cf6-1f14146ee1d0 None admin] RBAC: Adding query filter params (role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=8c495a22d3f7484cb377da68dab0bf97, user.id=8390e06ba7334f4d8e1afbf76a9b3591) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:48.653943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59846b-41c9-4182-8cf6-1f14146ee1d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:48.656688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59846b-41c9-4182-8cf6-1f14146ee1d0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:48.666877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 90/182] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:48 2022] GET /identity/v3/role_assignments?user.id=8390e06ba7334f4d8e1afbf76a9b3591&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:48.669587 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59846b-41c9-4182-8cf6-1f14146ee1d0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:48.687718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:48.692055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb21a2-ce28-47e6-a25f-b34db34e6bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:48.693658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb21a2-ce28-47e6-a25f-b34db34e6bec None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.694303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb21a2-ce28-47e6-a25f-b34db34e6bec None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.694842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb21a2-ce28-47e6-a25f-b34db34e6bec None admin] PATH_INFO: `/v3/projects/382dde80ea564adbbbe1676422e118d9` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.697276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb21a2-ce28-47e6-a25f-b34db34e6bec None admin] RBAC: Authorizing `identity:get_project(project_id=382dde80ea564adbbbe1676422e118d9)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:48.699697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb21a2-ce28-47e6-a25f-b34db34e6bec None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:48.702840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 93/183] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:48 2022] GET /identity/v3/projects/382dde80ea564adbbbe1676422e118d9 => generated 312 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:48.705195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887e89-9702-476d-804f-34b45bc831bf None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.705803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887e89-9702-476d-804f-34b45bc831bf None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.706370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887e89-9702-476d-804f-34b45bc831bf None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.709264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 94/184] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:48.710795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e887e89-9702-476d-804f-34b45bc831bf None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:48.905780 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e887e89-9702-476d-804f-34b45bc831bf None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:48.906505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59846b-41c9-4182-8cf6-1f14146ee1d0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:48.948114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:48.961727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73104f4-4ce8-4798-98a2-4edfdaf28e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:48.965157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73104f4-4ce8-4798-98a2-4edfdaf28e10 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.965157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73104f4-4ce8-4798-98a2-4edfdaf28e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.965157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73104f4-4ce8-4798-98a2-4edfdaf28e10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.972225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e08985-e226-449b-b063-97dc9f529a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:48.973164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e08985-e226-449b-b063-97dc9f529a62 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:48.973592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e08985-e226-449b-b063-97dc9f529a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:48.974560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e08985-e226-449b-b063-97dc9f529a62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:48.987727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73104f4-4ce8-4798-98a2-4edfdaf28e10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:48.990363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e08985-e226-449b-b063-97dc9f529a62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-46e08985-e226-449b-b063-97dc9f529a62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:48.999455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:49.001049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 91/185] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:14:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:49.002526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e08985-e226-449b-b063-97dc9f529a62 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-c73104f4-4ce8-4798-98a2-4edfdaf28e10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:49.007114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:49.008758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:49.010243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:49.014438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 95/186] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:49.017130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01995695-4ad1-4697-baf0-2ff614bf6c90 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.017928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01995695-4ad1-4697-baf0-2ff614bf6c90 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.018643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01995695-4ad1-4697-baf0-2ff614bf6c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.050706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-01995695-4ad1-4697-baf0-2ff614bf6c90 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:49.052911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-01995695-4ad1-4697-baf0-2ff614bf6c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:49.142164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:49.164166 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e08985-e226-449b-b063-97dc9f529a62 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:49.164727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 96/187] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:49.164727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01995695-4ad1-4697-baf0-2ff614bf6c90 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:49.227895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a29073-efd9-4d5c-9071-e0741241352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:49.228940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a29073-efd9-4d5c-9071-e0741241352e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.229400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a29073-efd9-4d5c-9071-e0741241352e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.230031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a29073-efd9-4d5c-9071-e0741241352e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.231575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a29073-efd9-4d5c-9071-e0741241352e None admin] RBAC: Adding query filter params (role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=382dde80ea564adbbbe1676422e118d9, user.id=8390e06ba7334f4d8e1afbf76a9b3591) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:49.232109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a29073-efd9-4d5c-9071-e0741241352e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:49.238557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a29073-efd9-4d5c-9071-e0741241352e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:49.245340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 92/188] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:49 2022] GET /identity/v3/role_assignments?user.id=8390e06ba7334f4d8e1afbf76a9b3591&scope.project.id=382dde80ea564adbbbe1676422e118d9&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:49.246760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a29073-efd9-4d5c-9071-e0741241352e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:49.347062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:49.353137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01995695-4ad1-4697-baf0-2ff614bf6c90 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:49.403681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108d0ec-18de-458d-b7c7-ee35623df668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:49.404558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108d0ec-18de-458d-b7c7-ee35623df668 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.405016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108d0ec-18de-458d-b7c7-ee35623df668 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.405472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108d0ec-18de-458d-b7c7-ee35623df668 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.406537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108d0ec-18de-458d-b7c7-ee35623df668 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:49.406984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108d0ec-18de-458d-b7c7-ee35623df668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:49.408835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108d0ec-18de-458d-b7c7-ee35623df668 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:49.423701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 97/189] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:14:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:49.423701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108d0ec-18de-458d-b7c7-ee35623df668 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:49.432123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a29073-efd9-4d5c-9071-e0741241352e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:49.495401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e135d-7d85-4d6f-be7e-f9926c2875ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:49.496134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e135d-7d85-4d6f-be7e-f9926c2875ab None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.496421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e135d-7d85-4d6f-be7e-f9926c2875ab None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.496710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e135d-7d85-4d6f-be7e-f9926c2875ab None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.497500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e135d-7d85-4d6f-be7e-f9926c2875ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:49.497813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e135d-7d85-4d6f-be7e-f9926c2875ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:49.499264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e135d-7d85-4d6f-be7e-f9926c2875ab None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:49.511674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:49.516313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 93/190] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:49.519095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f4fb2-5cf2-44df-a763-c5381f4386c5 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.523445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f4fb2-5cf2-44df-a763-c5381f4386c5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.523445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f4fb2-5cf2-44df-a763-c5381f4386c5 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.523445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 94/191] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:49.525659 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f4fb2-5cf2-44df-a763-c5381f4386c5 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:49.579392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108d0ec-18de-458d-b7c7-ee35623df668 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:49.618335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b70919-ee06-454a-aa72-894f916ac4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:49.619161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b70919-ee06-454a-aa72-894f916ac4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.619440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b70919-ee06-454a-aa72-894f916ac4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.619806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b70919-ee06-454a-aa72-894f916ac4e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.621123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b70919-ee06-454a-aa72-894f916ac4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:49.622623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b70919-ee06-454a-aa72-894f916ac4e5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:49.624404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 98/192] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:49.625254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b70919-ee06-454a-aa72-894f916ac4e5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:49.650849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f4fb2-5cf2-44df-a763-c5381f4386c5 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:49.659861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:49.693463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf98a03-fcac-4db2-ab1e-db8d546758b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:49.694370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf98a03-fcac-4db2-ab1e-db8d546758b1 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.694637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf98a03-fcac-4db2-ab1e-db8d546758b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.694910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf98a03-fcac-4db2-ab1e-db8d546758b1 None admin] PATH_INFO: `/v3/users/9d34ebe311ef4e78987ed667fe856e2d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.712687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf98a03-fcac-4db2-ab1e-db8d546758b1 None admin] RBAC: Authorizing `identity:get_user(user_id=9d34ebe311ef4e78987ed667fe856e2d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:49.714177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf98a03-fcac-4db2-ab1e-db8d546758b1 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:49.716025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 95/193] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:49 2022] GET /identity/v3/users/9d34ebe311ef4e78987ed667fe856e2d => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:49.717457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4535e3-c525-4d77-818f-e270059f73b5 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.717752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4535e3-c525-4d77-818f-e270059f73b5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.718026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4535e3-c525-4d77-818f-e270059f73b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.736419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-fa4535e3-c525-4d77-818f-e270059f73b5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:49.737654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fa4535e3-c525-4d77-818f-e270059f73b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:49.746161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b70919-ee06-454a-aa72-894f916ac4e5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:49.770854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:49.789137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0310d1-716c-4149-85f0-e4c7949b7f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:49.790199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0310d1-716c-4149-85f0-e4c7949b7f17 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.790661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0310d1-716c-4149-85f0-e4c7949b7f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.791181 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0310d1-716c-4149-85f0-e4c7949b7f17 None admin] PATH_INFO: `/v3/users/9d34ebe311ef4e78987ed667fe856e2d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.792869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0310d1-716c-4149-85f0-e4c7949b7f17 None admin] RBAC: Authorizing `identity:get_user(user_id=9d34ebe311ef4e78987ed667fe856e2d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:49.794615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0310d1-716c-4149-85f0-e4c7949b7f17 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:49.796929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 99/194] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:49 2022] GET /identity/v3/users/9d34ebe311ef4e78987ed667fe856e2d => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:49.798615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0310d1-716c-4149-85f0-e4c7949b7f17 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:49.825488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 96/195] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:49.826620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4535e3-c525-4d77-818f-e270059f73b5 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:49.914258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0310d1-716c-4149-85f0-e4c7949b7f17 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:49.949707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:49.951641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646169c-d6b2-48ad-a10e-fcdcea18881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:49.952339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646169c-d6b2-48ad-a10e-fcdcea18881b None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.952620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646169c-d6b2-48ad-a10e-fcdcea18881b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.952893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646169c-d6b2-48ad-a10e-fcdcea18881b None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:49.953630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646169c-d6b2-48ad-a10e-fcdcea18881b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:49.955130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646169c-d6b2-48ad-a10e-fcdcea18881b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:49.957619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4535e3-c525-4d77-818f-e270059f73b5 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:49.998444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f67aa5-c876-4528-b4bf-468d1344cc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:49.999105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f67aa5-c876-4528-b4bf-468d1344cc17 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:49.999383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f67aa5-c876-4528-b4bf-468d1344cc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:49.999692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f67aa5-c876-4528-b4bf-468d1344cc17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:50.072825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 100/196] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:14:49 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:14:50.075185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646169c-d6b2-48ad-a10e-fcdcea18881b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:50.077683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f67aa5-c876-4528-b4bf-468d1344cc17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:50.079324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f67aa5-c876-4528-b4bf-468d1344cc17 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:50.176347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:50.208438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:50.223465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 97/197] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:50.225361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f67aa5-c876-4528-b4bf-468d1344cc17 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:50.281811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646169c-d6b2-48ad-a10e-fcdcea18881b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:50.364275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46dae4-175f-457f-933b-106475970860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:50.365357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46dae4-175f-457f-933b-106475970860 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:50.365772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46dae4-175f-457f-933b-106475970860 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:50.366169 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46dae4-175f-457f-933b-106475970860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:50.404809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f67aa5-c876-4528-b4bf-468d1344cc17 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:50.458080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51963-108c-48d0-90a7-20692bee1d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:50.459919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51963-108c-48d0-90a7-20692bee1d53 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:50.460160 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51963-108c-48d0-90a7-20692bee1d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:50.460408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51963-108c-48d0-90a7-20692bee1d53 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:50.480087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a51963-108c-48d0-90a7-20692bee1d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-84a51963-108c-48d0-90a7-20692bee1d53 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:50.496149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:50.497734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:14:50.499154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:14:50.500255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 98/198] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:50.501133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46dae4-175f-457f-933b-106475970860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:50.501384 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51963-108c-48d0-90a7-20692bee1d53 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:50.502349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46dae4-175f-457f-933b-106475970860 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:50.570631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:50.617056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:50.642494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 101/199] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:50.643248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46dae4-175f-457f-933b-106475970860 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:50.717368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51963-108c-48d0-90a7-20692bee1d53 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:50.770748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46dae4-175f-457f-933b-106475970860 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:50.784020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cf128-c901-4d73-938a-7b1929dfd593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:50.785492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cf128-c901-4d73-938a-7b1929dfd593 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:50.785971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cf128-c901-4d73-938a-7b1929dfd593 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:50.786456 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cf128-c901-4d73-938a-7b1929dfd593 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:50.788926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cf128-c901-4d73-938a-7b1929dfd593 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:50.791769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cf128-c901-4d73-938a-7b1929dfd593 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:50.795388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 99/200] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:50 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:50.804724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cf128-c901-4d73-938a-7b1929dfd593 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:50.814177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce331-fa47-400c-b1b4-612cc50d55a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:50.814892 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce331-fa47-400c-b1b4-612cc50d55a6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:50.815178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce331-fa47-400c-b1b4-612cc50d55a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:50.815493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce331-fa47-400c-b1b4-612cc50d55a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:50.816397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce331-fa47-400c-b1b4-612cc50d55a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:50.816715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce331-fa47-400c-b1b4-612cc50d55a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:50.818721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce331-fa47-400c-b1b4-612cc50d55a6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:50.829611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 102/201] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:50.830664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce331-fa47-400c-b1b4-612cc50d55a6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:50.847189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:50.964734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce331-fa47-400c-b1b4-612cc50d55a6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:51.009220 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442411f-b535-4308-a594-b45c3499065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:51.010228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442411f-b535-4308-a594-b45c3499065a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:51.010731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442411f-b535-4308-a594-b45c3499065a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:51.011167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442411f-b535-4308-a594-b45c3499065a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:51.012257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442411f-b535-4308-a594-b45c3499065a None admin] RBAC: Adding query filter params (role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=04cc0f746b104684b7b6e33d52a38126, user.id=9d34ebe311ef4e78987ed667fe856e2d) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:51.012845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442411f-b535-4308-a594-b45c3499065a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:51.014517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442411f-b535-4308-a594-b45c3499065a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:51.015730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cf128-c901-4d73-938a-7b1929dfd593 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:51.022264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 103/202] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:50 2022] GET /identity/v3/role_assignments?user.id=9d34ebe311ef4e78987ed667fe856e2d&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:51.028657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442411f-b535-4308-a594-b45c3499065a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:51.067515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:51.089085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0a7c6-b3cb-43cd-832a-2c5381747906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:51.090721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0a7c6-b3cb-43cd-832a-2c5381747906 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:51.091272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0a7c6-b3cb-43cd-832a-2c5381747906 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:51.091802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0a7c6-b3cb-43cd-832a-2c5381747906 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:51.095424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0a7c6-b3cb-43cd-832a-2c5381747906 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:51.098655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0a7c6-b3cb-43cd-832a-2c5381747906 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:51.103638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 100/203] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:50 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:51.113985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0a7c6-b3cb-43cd-832a-2c5381747906 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:51.253544 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:51.336152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442411f-b535-4308-a594-b45c3499065a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:51.413249 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de84ab-76ff-4450-811d-c9bb3f705a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:51.414726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de84ab-76ff-4450-811d-c9bb3f705a28 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:51.415157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de84ab-76ff-4450-811d-c9bb3f705a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:51.415628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de84ab-76ff-4450-811d-c9bb3f705a28 None admin] PATH_INFO: `/v3/users/c9a693de281f41299e5061226360e7dd` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:51.421890 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0a7c6-b3cb-43cd-832a-2c5381747906 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:51.424339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de84ab-76ff-4450-811d-c9bb3f705a28 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a693de281f41299e5061226360e7dd)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:51.425908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de84ab-76ff-4450-811d-c9bb3f705a28 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:51.427783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 104/204] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:51 2022] GET /identity/v3/users/c9a693de281f41299e5061226360e7dd => generated 289 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:51.436463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de84ab-76ff-4450-811d-c9bb3f705a28 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:51.554184 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d1011-b777-4263-87d9-4cb733e2dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:51.555830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d1011-b777-4263-87d9-4cb733e2dd70 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:51.556608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d1011-b777-4263-87d9-4cb733e2dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:51.557365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d1011-b777-4263-87d9-4cb733e2dd70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:51.559321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d1011-b777-4263-87d9-4cb733e2dd70 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=04cc0f746b104684b7b6e33d52a38126, user.id=9d34ebe311ef4e78987ed667fe856e2d) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:51.560485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d1011-b777-4263-87d9-4cb733e2dd70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:51.563902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d1011-b777-4263-87d9-4cb733e2dd70 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:51.577338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 101/205] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:51 2022] GET /identity/v3/role_assignments?user.id=9d34ebe311ef4e78987ed667fe856e2d&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=03b724e03f704f2ab338b64025a3db6d => generated 259 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:51.588755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de84ab-76ff-4450-811d-c9bb3f705a28 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:51.661112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5370e-e425-477e-87d1-4b912c4cc064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:51.663372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5370e-e425-477e-87d1-4b912c4cc064 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:51.664143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5370e-e425-477e-87d1-4b912c4cc064 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:51.664907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5370e-e425-477e-87d1-4b912c4cc064 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:51.681925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:51.804300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5370e-e425-477e-87d1-4b912c4cc064 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:51.808212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5370e-e425-477e-87d1-4b912c4cc064 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:52.051854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 105/206] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:52.062216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d1011-b777-4263-87d9-4cb733e2dd70 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:52.209991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:52.278060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d1011-b777-4263-87d9-4cb733e2dd70 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:52.338361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889ef3f-4a98-46d4-8c1f-aae8de196057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:52.339179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889ef3f-4a98-46d4-8c1f-aae8de196057 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:52.339458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889ef3f-4a98-46d4-8c1f-aae8de196057 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:52.339753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889ef3f-4a98-46d4-8c1f-aae8de196057 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:52.341088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889ef3f-4a98-46d4-8c1f-aae8de196057 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:52.342560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889ef3f-4a98-46d4-8c1f-aae8de196057 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:52.344579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 102/207] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:52 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:52.350963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5370e-e425-477e-87d1-4b912c4cc064 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:52.464738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:52.621062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5370e-e425-477e-87d1-4b912c4cc064 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:52.719039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff79938-9a60-401e-8d0c-15c547996bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:52.720162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff79938-9a60-401e-8d0c-15c547996bcf None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:52.720650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff79938-9a60-401e-8d0c-15c547996bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:52.721266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff79938-9a60-401e-8d0c-15c547996bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:52.722913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff79938-9a60-401e-8d0c-15c547996bcf None admin] RBAC: Adding query filter params (role.id=536229a47a9d4a03b6d0b1c59fd8f39a, scope.project.id=8c495a22d3f7484cb377da68dab0bf97, user.id=c9a693de281f41299e5061226360e7dd) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:52.723461 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff79938-9a60-401e-8d0c-15c547996bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:52.726702 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff79938-9a60-401e-8d0c-15c547996bcf None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:52.744610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 106/208] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:52 2022] GET /identity/v3/role_assignments?user.id=c9a693de281f41299e5061226360e7dd&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=536229a47a9d4a03b6d0b1c59fd8f39a => generated 608 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:53.524688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabed25-1b31-40c9-b166-47f32d7540b8 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:53.525476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabed25-1b31-40c9-b166-47f32d7540b8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:53.525476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabed25-1b31-40c9-b166-47f32d7540b8 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:53.526736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 103/209] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:53.532544 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35f8cf-a6b4-409e-b7f9-66c2e829e4c9 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:53.533280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35f8cf-a6b4-409e-b7f9-66c2e829e4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:53.533280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35f8cf-a6b4-409e-b7f9-66c2e829e4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:53.552749 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7f35f8cf-a6b4-409e-b7f9-66c2e829e4c9 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:53.553990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7f35f8cf-a6b4-409e-b7f9-66c2e829e4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:53.624435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744665e-8ab8-46d4-969c-44a0c0867c70 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:53.624759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744665e-8ab8-46d4-969c-44a0c0867c70 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:53.625062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744665e-8ab8-46d4-969c-44a0c0867c70 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:53.626481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 104/210] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:53.633041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231645e-e053-499e-b8ef-e8f07e5e7da4 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:53.633441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231645e-e053-499e-b8ef-e8f07e5e7da4 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:53.633818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231645e-e053-499e-b8ef-e8f07e5e7da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:53.652232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 107/211] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:53.657470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9231645e-e053-499e-b8ef-e8f07e5e7da4 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:53.659766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9231645e-e053-499e-b8ef-e8f07e5e7da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:53.678466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35f8cf-a6b4-409e-b7f9-66c2e829e4c9 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:53.725337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:53.813868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 105/212] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:53.832141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35f8cf-a6b4-409e-b7f9-66c2e829e4c9 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:53.843186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231645e-e053-499e-b8ef-e8f07e5e7da4 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:53.883516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeebb95-93fa-4b0b-a9c6-fc74eea23bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:53.885497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeebb95-93fa-4b0b-a9c6-fc74eea23bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:53.886039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeebb95-93fa-4b0b-a9c6-fc74eea23bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:53.886541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeebb95-93fa-4b0b-a9c6-fc74eea23bd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:53.902566 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeebb95-93fa-4b0b-a9c6-fc74eea23bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-7eeebb95-93fa-4b0b-a9c6-fc74eea23bd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:53.922357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:53.923344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:53.924200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:53.931882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 108/213] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:14:53.934880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeebb95-93fa-4b0b-a9c6-fc74eea23bd3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:53.938493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:54.019282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231645e-e053-499e-b8ef-e8f07e5e7da4 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:54.079128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d7f5a-9f47-4cb5-99d8-806e46e4aa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:54.080421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d7f5a-9f47-4cb5-99d8-806e46e4aa22 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:54.080934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d7f5a-9f47-4cb5-99d8-806e46e4aa22 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:54.081515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d7f5a-9f47-4cb5-99d8-806e46e4aa22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:54.083049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d7f5a-9f47-4cb5-99d8-806e46e4aa22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:54.085434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d7f5a-9f47-4cb5-99d8-806e46e4aa22 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:54.087429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 106/214] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:54.095911 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d7f5a-9f47-4cb5-99d8-806e46e4aa22 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:54.155494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:54.179975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeebb95-93fa-4b0b-a9c6-fc74eea23bd3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:54.233287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a301e0-cd2a-4920-b69d-21a5be1ada1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:54.234350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a301e0-cd2a-4920-b69d-21a5be1ada1a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:54.234831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a301e0-cd2a-4920-b69d-21a5be1ada1a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:54.235329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a301e0-cd2a-4920-b69d-21a5be1ada1a None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:54.236479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a301e0-cd2a-4920-b69d-21a5be1ada1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:54.237012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a301e0-cd2a-4920-b69d-21a5be1ada1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:54.239056 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a301e0-cd2a-4920-b69d-21a5be1ada1a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:54.256679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 109/215] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:54.258387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a301e0-cd2a-4920-b69d-21a5be1ada1a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:54.262427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d7f5a-9f47-4cb5-99d8-806e46e4aa22 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:54.290765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:54.309730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4fdf6-fe2b-4ed3-ba59-252a537b9446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:54.310954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4fdf6-fe2b-4ed3-ba59-252a537b9446 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:54.311469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4fdf6-fe2b-4ed3-ba59-252a537b9446 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:54.312033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4fdf6-fe2b-4ed3-ba59-252a537b9446 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:54.312975 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4fdf6-fe2b-4ed3-ba59-252a537b9446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:54.314754 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4fdf6-fe2b-4ed3-ba59-252a537b9446 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:54.419751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 107/216] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:14:54 2022] POST /identity/v3/users => generated 297 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:14:54.478400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a301e0-cd2a-4920-b69d-21a5be1ada1a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:54.558082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698c611-893d-44fb-b6c8-0d8fc91f080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:54.559810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c611-893d-44fb-b6c8-0d8fc91f080c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:54.560331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c611-893d-44fb-b6c8-0d8fc91f080c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:54.560808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c611-893d-44fb-b6c8-0d8fc91f080c None admin] PATH_INFO: `/v3/users/6d0b9295aa42422184417453cd03267f` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:54.590993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698c611-893d-44fb-b6c8-0d8fc91f080c None admin] RBAC: Authorizing `identity:get_user(user_id=6d0b9295aa42422184417453cd03267f)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:54.594033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698c611-893d-44fb-b6c8-0d8fc91f080c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:54.597473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 110/217] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:54 2022] GET /identity/v3/users/6d0b9295aa42422184417453cd03267f => generated 297 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:54.604322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4fdf6-fe2b-4ed3-ba59-252a537b9446 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:54.718533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd77cd-4d3e-4566-8079-8c7d9c187a55 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:54.718948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd77cd-4d3e-4566-8079-8c7d9c187a55 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:54.719434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd77cd-4d3e-4566-8079-8c7d9c187a55 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:54.722254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 111/218] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:54.730672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4fdf6-fe2b-4ed3-ba59-252a537b9446 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:54.731973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeaad5-7c23-48d7-a6b1-a54b0b7009d5 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:54.732482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeaad5-7c23-48d7-a6b1-a54b0b7009d5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:54.732971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeaad5-7c23-48d7-a6b1-a54b0b7009d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:54.767125 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-52eeaad5-7c23-48d7-a6b1-a54b0b7009d5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:54.770189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-52eeaad5-7c23-48d7-a6b1-a54b0b7009d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:54.781328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bb6c0-2e38-4e87-bc86-eb688b62c0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:54.782995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bb6c0-2e38-4e87-bc86-eb688b62c0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:54.783452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bb6c0-2e38-4e87-bc86-eb688b62c0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:54.783905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bb6c0-2e38-4e87-bc86-eb688b62c0b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:54.785514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:54.858244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:54.895936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 112/219] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:54.904885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd108602-20fb-4adc-84c5-570d40eccbca None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:54.905459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd108602-20fb-4adc-84c5-570d40eccbca None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:54.905911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd108602-20fb-4adc-84c5-570d40eccbca None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:54.908028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 113/220] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:54.913128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bb6c0-2e38-4e87-bc86-eb688b62c0b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:54.914646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bb6c0-2e38-4e87-bc86-eb688b62c0b0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:54.917815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45159d-43f2-4715-9319-e98227484252 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:54.919170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45159d-43f2-4715-9319-e98227484252 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:54.919486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45159d-43f2-4715-9319-e98227484252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:54.941454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4e45159d-43f2-4715-9319-e98227484252 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:54.942737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4e45159d-43f2-4715-9319-e98227484252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:55.029306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 114/221] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:14:55.030484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45159d-43f2-4715-9319-e98227484252 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:55.062524 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 108/222] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:14:55.068776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bb6c0-2e38-4e87-bc86-eb688b62c0b0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:55.150765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:55.184026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:55.185381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45159d-43f2-4715-9319-e98227484252 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:55.210781 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bb6c0-2e38-4e87-bc86-eb688b62c0b0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:55.226784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c10911-33bf-437f-8a91-f0f89a8fc031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:55.227855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c10911-33bf-437f-8a91-f0f89a8fc031 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:55.228243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c10911-33bf-437f-8a91-f0f89a8fc031 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:55.228737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c10911-33bf-437f-8a91-f0f89a8fc031 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:55.241956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c10911-33bf-437f-8a91-f0f89a8fc031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-29c10911-33bf-437f-8a91-f0f89a8fc031 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:55.250890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:55.255986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 115/223] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:55.256771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c10911-33bf-437f-8a91-f0f89a8fc031 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:55.259589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac594c-a5c8-4463-b97e-0223fb55bd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:55.260799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac594c-a5c8-4463-b97e-0223fb55bd78 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:55.261393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac594c-a5c8-4463-b97e-0223fb55bd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:55.261951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac594c-a5c8-4463-b97e-0223fb55bd78 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:55.263914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac594c-a5c8-4463-b97e-0223fb55bd78 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:55.265887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac594c-a5c8-4463-b97e-0223fb55bd78 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:55.274946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 109/224] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:55.276293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac594c-a5c8-4463-b97e-0223fb55bd78 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:55.376647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c10911-33bf-437f-8a91-f0f89a8fc031 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:55.405971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac594c-a5c8-4463-b97e-0223fb55bd78 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:55.410665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:55.418371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24817e11-4829-4c26-9bb1-855d3deb9419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:55.419266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24817e11-4829-4c26-9bb1-855d3deb9419 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:55.419469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24817e11-4829-4c26-9bb1-855d3deb9419 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:55.419913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24817e11-4829-4c26-9bb1-855d3deb9419 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:55.433393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24817e11-4829-4c26-9bb1-855d3deb9419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-24817e11-4829-4c26-9bb1-855d3deb9419 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:55.441848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:55.443413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:55.444266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 116/225] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:55.444949 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24817e11-4829-4c26-9bb1-855d3deb9419 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:55.460275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af9bf-49e1-4602-9fa1-3af937168fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:55.461510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af9bf-49e1-4602-9fa1-3af937168fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:55.461990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af9bf-49e1-4602-9fa1-3af937168fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:55.462275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af9bf-49e1-4602-9fa1-3af937168fe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:55.463065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99af9bf-49e1-4602-9fa1-3af937168fe3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:55.463379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99af9bf-49e1-4602-9fa1-3af937168fe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:55.465076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99af9bf-49e1-4602-9fa1-3af937168fe3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:55.479567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 110/226] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:55.479567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af9bf-49e1-4602-9fa1-3af937168fe3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:55.487577 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:55.602059 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24817e11-4829-4c26-9bb1-855d3deb9419 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:55.621477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af9bf-49e1-4602-9fa1-3af937168fe3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:55.649767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bdadd-06c5-486d-8e9a-2c4165b63bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:55.650558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bdadd-06c5-486d-8e9a-2c4165b63bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:55.651432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bdadd-06c5-486d-8e9a-2c4165b63bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:55.651768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bdadd-06c5-486d-8e9a-2c4165b63bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:55.652819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bdadd-06c5-486d-8e9a-2c4165b63bf2 None admin] RBAC: Adding query filter params (role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=04cc0f746b104684b7b6e33d52a38126, user.id=6d0b9295aa42422184417453cd03267f) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:55.653283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bdadd-06c5-486d-8e9a-2c4165b63bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:55.655078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bdadd-06c5-486d-8e9a-2c4165b63bf2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:55.663358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 117/227] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/role_assignments?user.id=6d0b9295aa42422184417453cd03267f&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:55.664301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bdadd-06c5-486d-8e9a-2c4165b63bf2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:55.671568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4923e1-bcd4-4c5e-84d7-35a50ccff47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:55.672715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4923e1-bcd4-4c5e-84d7-35a50ccff47e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:55.673220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4923e1-bcd4-4c5e-84d7-35a50ccff47e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:55.673735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4923e1-bcd4-4c5e-84d7-35a50ccff47e None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:55.674760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4923e1-bcd4-4c5e-84d7-35a50ccff47e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:55.676118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4923e1-bcd4-4c5e-84d7-35a50ccff47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:55.677671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4923e1-bcd4-4c5e-84d7-35a50ccff47e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:55.702329 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 111/228] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:55.710518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4923e1-bcd4-4c5e-84d7-35a50ccff47e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:55.761098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:55.819820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bdadd-06c5-486d-8e9a-2c4165b63bf2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:55.825259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:55.880145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ad9d-caf0-4fa3-a931-010240a58e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:55.881607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ad9d-caf0-4fa3-a931-010240a58e6b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:55.882129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ad9d-caf0-4fa3-a931-010240a58e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:55.882626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ad9d-caf0-4fa3-a931-010240a58e6b None admin] PATH_INFO: `/v3/users/9d34ebe311ef4e78987ed667fe856e2d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:55.882842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4923e1-bcd4-4c5e-84d7-35a50ccff47e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:55.884648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8ad9d-caf0-4fa3-a931-010240a58e6b None admin] RBAC: Authorizing `identity:get_user(user_id=9d34ebe311ef4e78987ed667fe856e2d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:55.886466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8ad9d-caf0-4fa3-a931-010240a58e6b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:55.888684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 118/229] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/users/9d34ebe311ef4e78987ed667fe856e2d => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:55.894729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ad9d-caf0-4fa3-a931-010240a58e6b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:55.926757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aeac91-b985-4d62-b50a-7b9207f9437b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:55.927809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeac91-b985-4d62-b50a-7b9207f9437b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:55.928360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeac91-b985-4d62-b50a-7b9207f9437b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:55.928670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aeac91-b985-4d62-b50a-7b9207f9437b None admin] PATH_INFO: `/v3/users/9d34ebe311ef4e78987ed667fe856e2d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:55.930244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeac91-b985-4d62-b50a-7b9207f9437b None admin] RBAC: Authorizing `identity:get_user(user_id=9d34ebe311ef4e78987ed667fe856e2d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:55.931953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aeac91-b985-4d62-b50a-7b9207f9437b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:55.934230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 112/230] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/users/9d34ebe311ef4e78987ed667fe856e2d => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:55.949590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aeac91-b985-4d62-b50a-7b9207f9437b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:56.030350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ad9d-caf0-4fa3-a931-010240a58e6b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:56.060506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:56.074391 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8389adf-620d-4d3d-8a14-78d70ab11a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:56.075497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8389adf-620d-4d3d-8a14-78d70ab11a87 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:56.076004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8389adf-620d-4d3d-8a14-78d70ab11a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:56.076321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8389adf-620d-4d3d-8a14-78d70ab11a87 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:56.077900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8389adf-620d-4d3d-8a14-78d70ab11a87 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:56.079958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8389adf-620d-4d3d-8a14-78d70ab11a87 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:56.081810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 119/231] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:56.088281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8389adf-620d-4d3d-8a14-78d70ab11a87 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:56.096021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aeac91-b985-4d62-b50a-7b9207f9437b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:56.139508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46fe54-ffbb-4712-a4fb-51a37f96e4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:56.140727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46fe54-ffbb-4712-a4fb-51a37f96e4ab None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:56.141367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46fe54-ffbb-4712-a4fb-51a37f96e4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:56.141791 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46fe54-ffbb-4712-a4fb-51a37f96e4ab None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:56.143439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46fe54-ffbb-4712-a4fb-51a37f96e4ab None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:56.145260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46fe54-ffbb-4712-a4fb-51a37f96e4ab None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:56.148516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 113/232] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:14:55 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:56.157400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46fe54-ffbb-4712-a4fb-51a37f96e4ab None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:56.163961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:56.223972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8389adf-620d-4d3d-8a14-78d70ab11a87 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:56.272397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffc5c6-185f-4eb2-b00c-293b7009eaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:56.275386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffc5c6-185f-4eb2-b00c-293b7009eaad None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:56.276004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffc5c6-185f-4eb2-b00c-293b7009eaad None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:56.276489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffc5c6-185f-4eb2-b00c-293b7009eaad None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126/users/9d34ebe311ef4e78987ed667fe856e2d/roles/296546e6917a4eb6b58cac525a76f8a6` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:56.279400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffc5c6-185f-4eb2-b00c-293b7009eaad None admin] RBAC: Authorizing `identity:create_grant(project_id=04cc0f746b104684b7b6e33d52a38126, user_id=9d34ebe311ef4e78987ed667fe856e2d, role_id=296546e6917a4eb6b58cac525a76f8a6)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:56.281255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffc5c6-185f-4eb2-b00c-293b7009eaad None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:56.350601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46fe54-ffbb-4712-a4fb-51a37f96e4ab None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:56.381922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 120/233] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:14:56 2022] PUT /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126/users/9d34ebe311ef4e78987ed667fe856e2d/roles/296546e6917a4eb6b58cac525a76f8a6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:14:56.396756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebfa37-9244-4198-8ab3-f10c8c7d9548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:56.397835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebfa37-9244-4198-8ab3-f10c8c7d9548 None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:56.398120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebfa37-9244-4198-8ab3-f10c8c7d9548 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:56.398392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebfa37-9244-4198-8ab3-f10c8c7d9548 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126/users/9d34ebe311ef4e78987ed667fe856e2d/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:56.401065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebfa37-9244-4198-8ab3-f10c8c7d9548 None admin] RBAC: Authorizing `identity:create_grant(project_id=04cc0f746b104684b7b6e33d52a38126, user_id=9d34ebe311ef4e78987ed667fe856e2d, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:56.402785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebfa37-9244-4198-8ab3-f10c8c7d9548 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:56.514093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 114/234] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:14:56 2022] PUT /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126/users/9d34ebe311ef4e78987ed667fe856e2d/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:14:58.076286 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bd60e-5512-4352-be37-768cd8e8aeb0 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:58.077349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bd60e-5512-4352-be37-768cd8e8aeb0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:58.077860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bd60e-5512-4352-be37-768cd8e8aeb0 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:58.079983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 121/235] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:58.102772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221ff8-2921-4d41-a368-d18b2362ea0c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:58.103282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221ff8-2921-4d41-a368-d18b2362ea0c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:58.103752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221ff8-2921-4d41-a368-d18b2362ea0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:58.133852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd28d6-f704-494e-922b-c42e394ab672 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:58.134184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd28d6-f704-494e-922b-c42e394ab672 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:58.134410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd28d6-f704-494e-922b-c42e394ab672 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:58.136104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 122/236] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:58.141972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55698236-ef3f-4abc-8fa8-4b25538a746a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:58.142262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-5a221ff8-2921-4d41-a368-d18b2362ea0c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:58.142363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55698236-ef3f-4abc-8fa8-4b25538a746a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:58.142535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55698236-ef3f-4abc-8fa8-4b25538a746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:58.144622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5a221ff8-2921-4d41-a368-d18b2362ea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:58.175127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-55698236-ef3f-4abc-8fa8-4b25538a746a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:58.176504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-55698236-ef3f-4abc-8fa8-4b25538a746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:58.276158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:58.291511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 115/237] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:58.311308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:58.319986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a221ff8-2921-4d41-a368-d18b2362ea0c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:58.368674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 123/238] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:58.442398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a221ff8-2921-4d41-a368-d18b2362ea0c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:58.461485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55698236-ef3f-4abc-8fa8-4b25538a746a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:58.489051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8420ee7-749c-4998-9747-5f5bb6143181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:58.490427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8420ee7-749c-4998-9747-5f5bb6143181 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:58.490706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8420ee7-749c-4998-9747-5f5bb6143181 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:58.490980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8420ee7-749c-4998-9747-5f5bb6143181 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:58.492353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8420ee7-749c-4998-9747-5f5bb6143181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:58.493991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8420ee7-749c-4998-9747-5f5bb6143181 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:58.496137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 116/239] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:14:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:58.501827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8420ee7-749c-4998-9747-5f5bb6143181 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:58.576107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:58.620045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8420ee7-749c-4998-9747-5f5bb6143181 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:58.660800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d9bfa-b821-46b7-9b23-17b6dfaff232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:58.662015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d9bfa-b821-46b7-9b23-17b6dfaff232 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:58.662298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d9bfa-b821-46b7-9b23-17b6dfaff232 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:58.662586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d9bfa-b821-46b7-9b23-17b6dfaff232 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:58.663326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d9bfa-b821-46b7-9b23-17b6dfaff232 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:58.665149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d9bfa-b821-46b7-9b23-17b6dfaff232 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:58.727696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55698236-ef3f-4abc-8fa8-4b25538a746a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:58.743620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 117/240] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:14:58 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:14:58.824809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3b9d0-49be-4e1f-b1f7-9e5223d39aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:58.826306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3b9d0-49be-4e1f-b1f7-9e5223d39aef None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:58.826805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3b9d0-49be-4e1f-b1f7-9e5223d39aef None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:58.827287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3b9d0-49be-4e1f-b1f7-9e5223d39aef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:58.856700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3b9d0-49be-4e1f-b1f7-9e5223d39aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-f5a3b9d0-49be-4e1f-b1f7-9e5223d39aef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:58.872560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:58.875122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:58.877232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 124/241] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:14:58.885343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d9bfa-b821-46b7-9b23-17b6dfaff232 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:59.108732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7c89d-c393-422b-bc1d-3dafc2e79add None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:59.109429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7c89d-c393-422b-bc1d-3dafc2e79add None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:59.109429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7c89d-c393-422b-bc1d-3dafc2e79add None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:59.111046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 125/242] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:14:59.116809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d9bfa-b821-46b7-9b23-17b6dfaff232 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:59.117433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446bf78-73f5-46b1-b412-df5470dfe346 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:59.117433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446bf78-73f5-46b1-b412-df5470dfe346 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:59.117722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446bf78-73f5-46b1-b412-df5470dfe346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:59.138258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8446bf78-73f5-46b1-b412-df5470dfe346 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:59.139445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8446bf78-73f5-46b1-b412-df5470dfe346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:59.154837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:59.193605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:59.206253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d028fa3-1141-4563-8fbe-2ab5a136dd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:59.207857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d028fa3-1141-4563-8fbe-2ab5a136dd0d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:59.209275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d028fa3-1141-4563-8fbe-2ab5a136dd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:59.210926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d028fa3-1141-4563-8fbe-2ab5a136dd0d None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:59.212700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d028fa3-1141-4563-8fbe-2ab5a136dd0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:14:59.213444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d028fa3-1141-4563-8fbe-2ab5a136dd0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:59.217287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d028fa3-1141-4563-8fbe-2ab5a136dd0d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:59.242196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 118/243] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:14:59.252301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d028fa3-1141-4563-8fbe-2ab5a136dd0d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:59.255805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 126/244] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:59.282115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446bf78-73f5-46b1-b412-df5470dfe346 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:59.416036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d028fa3-1141-4563-8fbe-2ab5a136dd0d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:59.461014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ece54-06af-4587-a809-c8c98b3844dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:59.462364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ece54-06af-4587-a809-c8c98b3844dd None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:59.462805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ece54-06af-4587-a809-c8c98b3844dd None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:59.463251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ece54-06af-4587-a809-c8c98b3844dd None admin] PATH_INFO: `/v3/users/590e237b883045aa84ae9823fa39391f` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:59.480685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ece54-06af-4587-a809-c8c98b3844dd None admin] RBAC: Authorizing `identity:get_user(user_id=590e237b883045aa84ae9823fa39391f)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:59.482557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ece54-06af-4587-a809-c8c98b3844dd None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:14:59.484642 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 119/245] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:14:59 2022] GET /identity/v3/users/590e237b883045aa84ae9823fa39391f => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:14:59.498043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ece54-06af-4587-a809-c8c98b3844dd None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:59.522286 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446bf78-73f5-46b1-b412-df5470dfe346 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:59.549273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:59.595567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c43ac-499d-471f-9355-d9397b98efac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:59.597376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c43ac-499d-471f-9355-d9397b98efac None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:59.598098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c43ac-499d-471f-9355-d9397b98efac None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:59.598944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c43ac-499d-471f-9355-d9397b98efac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:59.622437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c43ac-499d-471f-9355-d9397b98efac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:14:59.628819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ece54-06af-4587-a809-c8c98b3844dd None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-941c43ac-499d-471f-9355-d9397b98efac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:14:59.638742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:14:59.639865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:14:59.640661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:14:59.641735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 127/246] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:14:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:14:59.643831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c88f-7af8-4eb7-9b1a-1ee227aba76a None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:59.644313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c88f-7af8-4eb7-9b1a-1ee227aba76a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:59.644789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c88f-7af8-4eb7-9b1a-1ee227aba76a None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:59.647971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 128/247] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:14:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:14:59.656215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84649c-e729-41e6-9ca9-c0c00ae9fd08 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:59.656821 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84649c-e729-41e6-9ca9-c0c00ae9fd08 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:59.657393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84649c-e729-41e6-9ca9-c0c00ae9fd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:59.682297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fcd75-494c-44f2-ae35-93df0cdb6ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:14:59.683264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fcd75-494c-44f2-ae35-93df0cdb6ebb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:14:59.683777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fcd75-494c-44f2-ae35-93df0cdb6ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:14:59.684234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fcd75-494c-44f2-ae35-93df0cdb6ebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:14:59.685922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-bd84649c-e729-41e6-9ca9-c0c00ae9fd08 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:14:59.687160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-bd84649c-e729-41e6-9ca9-c0c00ae9fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:14:59.736876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:14:59.811321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 129/248] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:14:59.819405 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84649c-e729-41e6-9ca9-c0c00ae9fd08 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:14:59.881911 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fcd75-494c-44f2-ae35-93df0cdb6ebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:14:59.884871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fcd75-494c-44f2-ae35-93df0cdb6ebb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:00.004467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:00.062307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84649c-e729-41e6-9ca9-c0c00ae9fd08 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:00.099127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 120/249] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:00.100137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fcd75-494c-44f2-ae35-93df0cdb6ebb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:00.122009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5e409-6ecb-4b7b-b063-9656d8ba85e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:00.124274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5e409-6ecb-4b7b-b063-9656d8ba85e1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:00.124837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5e409-6ecb-4b7b-b063-9656d8ba85e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:00.125321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5e409-6ecb-4b7b-b063-9656d8ba85e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:00.126857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5e409-6ecb-4b7b-b063-9656d8ba85e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:00.127432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5e409-6ecb-4b7b-b063-9656d8ba85e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:00.129201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5e409-6ecb-4b7b-b063-9656d8ba85e1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:00.144358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 130/250] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:14:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:00.151344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecc907-9257-442d-b608-7a44d0516349 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:00.151994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecc907-9257-442d-b608-7a44d0516349 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:00.152505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecc907-9257-442d-b608-7a44d0516349 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:00.154244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 131/251] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:00.155322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecc907-9257-442d-b608-7a44d0516349 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:00.207942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:00.250773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fcd75-494c-44f2-ae35-93df0cdb6ebb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:00.322085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f871b1-27a1-4e41-9f33-7ba3f5a36cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:00.323003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f871b1-27a1-4e41-9f33-7ba3f5a36cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:00.323291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f871b1-27a1-4e41-9f33-7ba3f5a36cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:00.323590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f871b1-27a1-4e41-9f33-7ba3f5a36cd7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:00.334994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:00.341452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f871b1-27a1-4e41-9f33-7ba3f5a36cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-c3f871b1-27a1-4e41-9f33-7ba3f5a36cd7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:00.352191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:00.355860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:00.357640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:00.358766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecc907-9257-442d-b608-7a44d0516349 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:00.358896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 121/252] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:15:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:00.358896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f871b1-27a1-4e41-9f33-7ba3f5a36cd7 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:00.430433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69da119-a1c4-4d1b-9f4d-bf500d0d53a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:00.432331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69da119-a1c4-4d1b-9f4d-bf500d0d53a8 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:00.432911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69da119-a1c4-4d1b-9f4d-bf500d0d53a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:00.433486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69da119-a1c4-4d1b-9f4d-bf500d0d53a8 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:00.435608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69da119-a1c4-4d1b-9f4d-bf500d0d53a8 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:00.438529 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69da119-a1c4-4d1b-9f4d-bf500d0d53a8 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:00.442195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 132/253] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:00 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:00.452786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffaa16-edf1-4658-bf2b-b3ba58513ae3 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:00.453381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffaa16-edf1-4658-bf2b-b3ba58513ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:00.454585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffaa16-edf1-4658-bf2b-b3ba58513ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:00.486298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2bffaa16-edf1-4658-bf2b-b3ba58513ae3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:00.488653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2bffaa16-edf1-4658-bf2b-b3ba58513ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:00.550021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f871b1-27a1-4e41-9f33-7ba3f5a36cd7 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:00.611432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6173-dad9-42cc-9cd4-e9f32df55b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:00.613330 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6173-dad9-42cc-9cd4-e9f32df55b04 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:00.614190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6173-dad9-42cc-9cd4-e9f32df55b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:00.614965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6173-dad9-42cc-9cd4-e9f32df55b04 None admin] PATH_INFO: `/v3/users/6d0b9295aa42422184417453cd03267f` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:00.617259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba6173-dad9-42cc-9cd4-e9f32df55b04 None admin] RBAC: Authorizing `identity:get_user(user_id=6d0b9295aa42422184417453cd03267f)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:00.619164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba6173-dad9-42cc-9cd4-e9f32df55b04 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:00.622656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 122/254] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:00 2022] GET /identity/v3/users/6d0b9295aa42422184417453cd03267f => generated 297 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:00.624721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6173-dad9-42cc-9cd4-e9f32df55b04 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:00.627415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 133/255] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:00.629181 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffaa16-edf1-4658-bf2b-b3ba58513ae3 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:00.678954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:00.732229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:00.767442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6173-dad9-42cc-9cd4-e9f32df55b04 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:00.826058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cc681-3ab6-4baa-9c0c-2433a6895d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:00.826861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cc681-3ab6-4baa-9c0c-2433a6895d19 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:00.827251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cc681-3ab6-4baa-9c0c-2433a6895d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:00.827676 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cc681-3ab6-4baa-9c0c-2433a6895d19 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:00.828621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cc681-3ab6-4baa-9c0c-2433a6895d19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:00.829213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cc681-3ab6-4baa-9c0c-2433a6895d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:00.830669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cc681-3ab6-4baa-9c0c-2433a6895d19 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:00.843592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 123/256] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:00.843592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cc681-3ab6-4baa-9c0c-2433a6895d19 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:00.851212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffaa16-edf1-4658-bf2b-b3ba58513ae3 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:00.898828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53aca4-471e-41ca-b287-8905126e8ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:00.899827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53aca4-471e-41ca-b287-8905126e8ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:00.900113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53aca4-471e-41ca-b287-8905126e8ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:00.900395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53aca4-471e-41ca-b287-8905126e8ae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:00.901322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53aca4-471e-41ca-b287-8905126e8ae5 None admin] RBAC: Adding query filter params (role.id=536229a47a9d4a03b6d0b1c59fd8f39a, scope.project.id=04cc0f746b104684b7b6e33d52a38126, user.id=590e237b883045aa84ae9823fa39391f) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:00.901695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53aca4-471e-41ca-b287-8905126e8ae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:00.903270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53aca4-471e-41ca-b287-8905126e8ae5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:00.915568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 134/257] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:00 2022] GET /identity/v3/role_assignments?user.id=590e237b883045aa84ae9823fa39391f&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=536229a47a9d4a03b6d0b1c59fd8f39a => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:00.915568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53aca4-471e-41ca-b287-8905126e8ae5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:01.099921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cc681-3ab6-4baa-9c0c-2433a6895d19 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:01.164281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:01.182850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4f970-daf7-4ec6-8153-719a07dd7833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:01.184832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4f970-daf7-4ec6-8153-719a07dd7833 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:01.185318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4f970-daf7-4ec6-8153-719a07dd7833 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:01.185803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4f970-daf7-4ec6-8153-719a07dd7833 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:01.189312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:01.189993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4f970-daf7-4ec6-8153-719a07dd7833 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:01.196509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4f970-daf7-4ec6-8153-719a07dd7833 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:01.199625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 124/258] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:00 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:01.200996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4f970-daf7-4ec6-8153-719a07dd7833 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:01.229213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53aca4-471e-41ca-b287-8905126e8ae5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:01.353939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122c61a-1cf0-4202-8295-4141ca1d24ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:01.354700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122c61a-1cf0-4202-8295-4141ca1d24ae None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:01.354988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122c61a-1cf0-4202-8295-4141ca1d24ae None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:01.355270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122c61a-1cf0-4202-8295-4141ca1d24ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:01.392740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122c61a-1cf0-4202-8295-4141ca1d24ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-6122c61a-1cf0-4202-8295-4141ca1d24ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:01.405759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:01.406735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:15:01.407514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:01.408310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 135/259] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:15:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 497 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:01.414003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122c61a-1cf0-4202-8295-4141ca1d24ae None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:01.445856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4f970-daf7-4ec6-8153-719a07dd7833 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:01.511988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d6481-54e7-44cc-a31b-a80525e3a5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:01.513648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6481-54e7-44cc-a31b-a80525e3a5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:01.514145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6481-54e7-44cc-a31b-a80525e3a5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:01.514622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d6481-54e7-44cc-a31b-a80525e3a5a2 None admin] PATH_INFO: `/v3/users/9d34ebe311ef4e78987ed667fe856e2d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:01.517236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d6481-54e7-44cc-a31b-a80525e3a5a2 None admin] RBAC: Authorizing `identity:get_user(user_id=9d34ebe311ef4e78987ed667fe856e2d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:01.520295 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d6481-54e7-44cc-a31b-a80525e3a5a2 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:01.523852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 125/260] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:01 2022] GET /identity/v3/users/9d34ebe311ef4e78987ed667fe856e2d => generated 283 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:01.525480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d6481-54e7-44cc-a31b-a80525e3a5a2 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:01.659001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122c61a-1cf0-4202-8295-4141ca1d24ae None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:01.708790 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81497359-5bf5-4069-8a99-f6f0d264a2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:01.710395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81497359-5bf5-4069-8a99-f6f0d264a2bc None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:01.710945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81497359-5bf5-4069-8a99-f6f0d264a2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:01.711498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81497359-5bf5-4069-8a99-f6f0d264a2bc None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126/users/6d0b9295aa42422184417453cd03267f/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:01.718827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81497359-5bf5-4069-8a99-f6f0d264a2bc None admin] RBAC: Authorizing `identity:create_grant(project_id=04cc0f746b104684b7b6e33d52a38126, user_id=6d0b9295aa42422184417453cd03267f, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:01.725514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81497359-5bf5-4069-8a99-f6f0d264a2bc None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:01.774408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:01.833585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 136/261] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:15:01 2022] PUT /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126/users/6d0b9295aa42422184417453cd03267f/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:01.837238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81497359-5bf5-4069-8a99-f6f0d264a2bc None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:01.873065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:01.878036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d6481-54e7-44cc-a31b-a80525e3a5a2 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:02.001026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fb145-3b7b-4b97-8117-410afcd75115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:02.002499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fb145-3b7b-4b97-8117-410afcd75115 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:02.003000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fb145-3b7b-4b97-8117-410afcd75115 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:02.003502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208fb145-3b7b-4b97-8117-410afcd75115 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:02.005013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fb145-3b7b-4b97-8117-410afcd75115 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:02.005557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fb145-3b7b-4b97-8117-410afcd75115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:02.008578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208fb145-3b7b-4b97-8117-410afcd75115 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:02.047565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 126/262] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:02.057064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fb145-3b7b-4b97-8117-410afcd75115 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:02.168207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81497359-5bf5-4069-8a99-f6f0d264a2bc None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:02.256589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353e569-2751-45a8-a52a-6be7f0911002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:02.258104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353e569-2751-45a8-a52a-6be7f0911002 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:02.258638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353e569-2751-45a8-a52a-6be7f0911002 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:02.259124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353e569-2751-45a8-a52a-6be7f0911002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:02.302972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208fb145-3b7b-4b97-8117-410afcd75115 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:02.396513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:02.424797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f51f6-824b-43c6-8355-2f178cde628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:02.426288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f51f6-824b-43c6-8355-2f178cde628c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:02.426779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f51f6-824b-43c6-8355-2f178cde628c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:02.427255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f51f6-824b-43c6-8355-2f178cde628c None admin] PATH_INFO: `/v3/users/9d34ebe311ef4e78987ed667fe856e2d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:02.433280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f51f6-824b-43c6-8355-2f178cde628c None admin] RBAC: Authorizing `identity:get_user(user_id=9d34ebe311ef4e78987ed667fe856e2d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:02.436163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f51f6-824b-43c6-8355-2f178cde628c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:02.436431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353e569-2751-45a8-a52a-6be7f0911002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:02.439458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353e569-2751-45a8-a52a-6be7f0911002 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:02.448086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 127/263] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:02 2022] GET /identity/v3/users/9d34ebe311ef4e78987ed667fe856e2d => generated 283 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:02.450758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a447e-8d13-45b9-ac62-ed7c73ef7c91 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:02.451262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a447e-8d13-45b9-ac62-ed7c73ef7c91 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:02.451809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a447e-8d13-45b9-ac62-ed7c73ef7c91 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:02.454491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 128/264] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:02.469870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a447e-8d13-45b9-ac62-ed7c73ef7c91 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:02.472787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:02.698765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 137/265] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 864 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:02.701196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde25b75-333f-4fb1-af53-7da38175c7ea None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:02.701478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde25b75-333f-4fb1-af53-7da38175c7ea None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:02.701754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde25b75-333f-4fb1-af53-7da38175c7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:02.732747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a447e-8d13-45b9-ac62-ed7c73ef7c91 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:02.733925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-fde25b75-333f-4fb1-af53-7da38175c7ea None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:02.735153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fde25b75-333f-4fb1-af53-7da38175c7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:02.869287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadb4d9-5fb1-4d4c-aea3-9dc43ab61f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:02.869973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadb4d9-5fb1-4d4c-aea3-9dc43ab61f07 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:02.870382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadb4d9-5fb1-4d4c-aea3-9dc43ab61f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:02.870662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadb4d9-5fb1-4d4c-aea3-9dc43ab61f07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:02.935662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 138/266] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:02.937414 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde25b75-333f-4fb1-af53-7da38175c7ea None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:03.036680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:03.105184 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadb4d9-5fb1-4d4c-aea3-9dc43ab61f07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:03.108400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:03.115998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadb4d9-5fb1-4d4c-aea3-9dc43ab61f07 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:03.179269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde25b75-333f-4fb1-af53-7da38175c7ea None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:03.255596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338b924-4f1c-4b18-94e4-60d20ae80ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:03.257501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338b924-4f1c-4b18-94e4-60d20ae80ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:03.259269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338b924-4f1c-4b18-94e4-60d20ae80ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:03.260266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338b924-4f1c-4b18-94e4-60d20ae80ae2 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:03.262912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338b924-4f1c-4b18-94e4-60d20ae80ae2 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:03.266500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338b924-4f1c-4b18-94e4-60d20ae80ae2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:03.270675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 139/267] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:02 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:03.279590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338b924-4f1c-4b18-94e4-60d20ae80ae2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:03.428955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 129/268] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 960 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 09:15:03.430668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadb4d9-5fb1-4d4c-aea3-9dc43ab61f07 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:03.500054 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338b924-4f1c-4b18-94e4-60d20ae80ae2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:03.505570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:03.580018 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd544e-69e8-42a2-8cbc-b9dff8dc60a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:03.581739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd544e-69e8-42a2-8cbc-b9dff8dc60a6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:03.582455 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd544e-69e8-42a2-8cbc-b9dff8dc60a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:03.584100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd544e-69e8-42a2-8cbc-b9dff8dc60a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:03.586926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd544e-69e8-42a2-8cbc-b9dff8dc60a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:03.590605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd544e-69e8-42a2-8cbc-b9dff8dc60a6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:03.594491 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 140/269] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:03.597296 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed6c5b-e998-4b37-945b-75d22ededb16 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:03.598111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed6c5b-e998-4b37-945b-75d22ededb16 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:03.598978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed6c5b-e998-4b37-945b-75d22ededb16 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:03.602378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 141/270] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:03.604425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed6c5b-e998-4b37-945b-75d22ededb16 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:03.637255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:03.650507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadb4d9-5fb1-4d4c-aea3-9dc43ab61f07 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:03.718533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73a93f-681d-424c-9665-b9b01e45cf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:03.720229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73a93f-681d-424c-9665-b9b01e45cf60 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:03.721026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73a93f-681d-424c-9665-b9b01e45cf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:03.721800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73a93f-681d-424c-9665-b9b01e45cf60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:03.723896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73a93f-681d-424c-9665-b9b01e45cf60 None admin] RBAC: Adding query filter params (role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=04cc0f746b104684b7b6e33d52a38126, user.id=9d34ebe311ef4e78987ed667fe856e2d) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:03.724756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73a93f-681d-424c-9665-b9b01e45cf60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:03.727939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73a93f-681d-424c-9665-b9b01e45cf60 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:03.740341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 130/271] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:03 2022] GET /identity/v3/role_assignments?user.id=9d34ebe311ef4e78987ed667fe856e2d&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:03.742270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73a93f-681d-424c-9665-b9b01e45cf60 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:03.827498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed6c5b-e998-4b37-945b-75d22ededb16 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:03.903303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a7dcd-67a1-4779-93ad-5601e813c632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:03.905479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a7dcd-67a1-4779-93ad-5601e813c632 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:03.906339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a7dcd-67a1-4779-93ad-5601e813c632 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:03.907138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a7dcd-67a1-4779-93ad-5601e813c632 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:03.910026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a7dcd-67a1-4779-93ad-5601e813c632 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:03.914049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a7dcd-67a1-4779-93ad-5601e813c632 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:03.923910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 142/272] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:03 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:03.926665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288568da-3da8-47e8-a0a0-4e85dd54235e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:03.927489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288568da-3da8-47e8-a0a0-4e85dd54235e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:03.928349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288568da-3da8-47e8-a0a0-4e85dd54235e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:03.965119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-288568da-3da8-47e8-a0a0-4e85dd54235e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:03.967232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-288568da-3da8-47e8-a0a0-4e85dd54235e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:03.985887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73a93f-681d-424c-9665-b9b01e45cf60 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:04.041938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:04.050623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b20263-383d-4086-9602-a7d67029d75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:04.052743 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b20263-383d-4086-9602-a7d67029d75f None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:04.053522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b20263-383d-4086-9602-a7d67029d75f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:04.054249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b20263-383d-4086-9602-a7d67029d75f None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:04.055932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b20263-383d-4086-9602-a7d67029d75f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:04.058916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b20263-383d-4086-9602-a7d67029d75f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:04.094674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 143/273] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:04.097084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13485-6c2e-47db-9ef9-469e3b2a5d79 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:04.097599 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13485-6c2e-47db-9ef9-469e3b2a5d79 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:04.098142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13485-6c2e-47db-9ef9-469e3b2a5d79 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:04.106024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 144/274] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:04.106024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13485-6c2e-47db-9ef9-469e3b2a5d79 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:04.224699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 131/275] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:15:03 2022] POST /identity/v3/users => generated 293 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:15:04.227940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca11c6-6220-412f-8964-013cf70b5e08 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:04.228501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca11c6-6220-412f-8964-013cf70b5e08 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:04.229016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca11c6-6220-412f-8964-013cf70b5e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:04.255700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13485-6c2e-47db-9ef9-469e3b2a5d79 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:04.259634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f2ca11c6-6220-412f-8964-013cf70b5e08 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:04.261712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f2ca11c6-6220-412f-8964-013cf70b5e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:04.283045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:04.316406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270f97e-60ea-4fa1-bbe7-e19e002eefa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:04.317163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270f97e-60ea-4fa1-bbe7-e19e002eefa6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:04.317539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270f97e-60ea-4fa1-bbe7-e19e002eefa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:04.317786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270f97e-60ea-4fa1-bbe7-e19e002eefa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:04.318747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270f97e-60ea-4fa1-bbe7-e19e002eefa6 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=04cc0f746b104684b7b6e33d52a38126, user.id=9d34ebe311ef4e78987ed667fe856e2d) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:04.319071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270f97e-60ea-4fa1-bbe7-e19e002eefa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:04.320794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270f97e-60ea-4fa1-bbe7-e19e002eefa6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:04.330106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 145/276] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:04 2022] GET /identity/v3/role_assignments?user.id=9d34ebe311ef4e78987ed667fe856e2d&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=03b724e03f704f2ab338b64025a3db6d => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:04.330106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270f97e-60ea-4fa1-bbe7-e19e002eefa6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:04.421800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:04.441082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 132/277] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:04.491836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca11c6-6220-412f-8964-013cf70b5e08 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:04.537469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270f97e-60ea-4fa1-bbe7-e19e002eefa6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:04.590443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020f05e-4bd4-45dc-be68-669387203548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:04.591332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020f05e-4bd4-45dc-be68-669387203548 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:04.591722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020f05e-4bd4-45dc-be68-669387203548 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:04.592020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020f05e-4bd4-45dc-be68-669387203548 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:04.613124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020f05e-4bd4-45dc-be68-669387203548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-2020f05e-4bd4-45dc-be68-669387203548 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:04.623589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:04.625071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:04.626721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 146/278] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:04.633284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020f05e-4bd4-45dc-be68-669387203548 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:04.701668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca11c6-6220-412f-8964-013cf70b5e08 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:04.751714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7006954-d3fb-432d-ab74-29da5ff6a151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:04.752878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006954-d3fb-432d-ab74-29da5ff6a151 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:04.753196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006954-d3fb-432d-ab74-29da5ff6a151 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:04.754253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006954-d3fb-432d-ab74-29da5ff6a151 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:04.774044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006954-d3fb-432d-ab74-29da5ff6a151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-b7006954-d3fb-432d-ab74-29da5ff6a151 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:04.782656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:04.783660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:04.785028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 133/279] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:04.786948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3f1fe-b068-4192-835f-b502425dae1c None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:04.786948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3f1fe-b068-4192-835f-b502425dae1c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:04.786948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3f1fe-b068-4192-835f-b502425dae1c None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:04.788482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 134/280] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:04.795298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3f1fe-b068-4192-835f-b502425dae1c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:04.798827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:04.800239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:04.810775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020f05e-4bd4-45dc-be68-669387203548 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:04.860727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c59d5a-6d99-4682-93c3-4744874fd70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:04.861809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c59d5a-6d99-4682-93c3-4744874fd70c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:04.862422 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c59d5a-6d99-4682-93c3-4744874fd70c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:04.862878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c59d5a-6d99-4682-93c3-4744874fd70c None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:04.863964 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c59d5a-6d99-4682-93c3-4744874fd70c None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:04.864981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c59d5a-6d99-4682-93c3-4744874fd70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:04.866832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c59d5a-6d99-4682-93c3-4744874fd70c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:04.877859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 147/281] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:04.879600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ce6af-7806-4633-8bad-a62255bf37cb None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:04.880678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ce6af-7806-4633-8bad-a62255bf37cb None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:04.881134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ce6af-7806-4633-8bad-a62255bf37cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:04.903325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-915ce6af-7806-4633-8bad-a62255bf37cb None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:04.904988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-915ce6af-7806-4633-8bad-a62255bf37cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:04.965179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3f1fe-b068-4192-835f-b502425dae1c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:05.011925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 148/282] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:05.013922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ce6af-7806-4633-8bad-a62255bf37cb None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:05.015242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3c77c-dc53-4286-b635-775dd93eff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:05.016188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3c77c-dc53-4286-b635-775dd93eff06 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:05.016482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3c77c-dc53-4286-b635-775dd93eff06 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:05.016766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3c77c-dc53-4286-b635-775dd93eff06 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:05.017555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3c77c-dc53-4286-b635-775dd93eff06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:05.017870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3c77c-dc53-4286-b635-775dd93eff06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:05.023618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3c77c-dc53-4286-b635-775dd93eff06 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:05.032925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 135/283] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:05.039765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3c77c-dc53-4286-b635-775dd93eff06 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:05.153216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:05.190681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3c77c-dc53-4286-b635-775dd93eff06 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:05.226101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:05.248774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ce6af-7806-4633-8bad-a62255bf37cb None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:05.255420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946eed8-9a6d-4811-8127-52cafb96f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:05.257022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946eed8-9a6d-4811-8127-52cafb96f35e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:05.257521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946eed8-9a6d-4811-8127-52cafb96f35e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:05.258008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946eed8-9a6d-4811-8127-52cafb96f35e None admin] PATH_INFO: `/v3/users/590e237b883045aa84ae9823fa39391f` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:05.260451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946eed8-9a6d-4811-8127-52cafb96f35e None admin] RBAC: Authorizing `identity:get_user(user_id=590e237b883045aa84ae9823fa39391f)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:05.263327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946eed8-9a6d-4811-8127-52cafb96f35e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:05.267098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 136/284] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:05 2022] GET /identity/v3/users/590e237b883045aa84ae9823fa39391f => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:05.269403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946eed8-9a6d-4811-8127-52cafb96f35e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:05.304365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218baeb-8ee7-49f9-a64c-2cb0c8b353ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:05.305256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218baeb-8ee7-49f9-a64c-2cb0c8b353ff None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:05.305726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218baeb-8ee7-49f9-a64c-2cb0c8b353ff None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:05.306147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218baeb-8ee7-49f9-a64c-2cb0c8b353ff None admin] PATH_INFO: `/v3/users/91580c57f0ee458c8aded161c84b7392` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:05.326922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218baeb-8ee7-49f9-a64c-2cb0c8b353ff None admin] RBAC: Authorizing `identity:get_user(user_id=91580c57f0ee458c8aded161c84b7392)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:05.328845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218baeb-8ee7-49f9-a64c-2cb0c8b353ff None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:05.331365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 149/285] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:05 2022] GET /identity/v3/users/91580c57f0ee458c8aded161c84b7392 => generated 293 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:05.332466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218baeb-8ee7-49f9-a64c-2cb0c8b353ff None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:05.448500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946eed8-9a6d-4811-8127-52cafb96f35e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:05.498279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483d8ad-e8c0-4afe-adb6-482a86d9f0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:05.499100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483d8ad-e8c0-4afe-adb6-482a86d9f0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:05.499385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483d8ad-e8c0-4afe-adb6-482a86d9f0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:05.499760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483d8ad-e8c0-4afe-adb6-482a86d9f0f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:05.502023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:05.523812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483d8ad-e8c0-4afe-adb6-482a86d9f0f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:05.528393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218baeb-8ee7-49f9-a64c-2cb0c8b353ff None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-7483d8ad-e8c0-4afe-adb6-482a86d9f0f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:05.540599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:05.541733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:05.542601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:05.543215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 137/286] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:05.549588 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483d8ad-e8c0-4afe-adb6-482a86d9f0f6 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:05.574722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6890863-883d-4b71-beed-f1d999dbd2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:05.575560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6890863-883d-4b71-beed-f1d999dbd2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:05.575862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6890863-883d-4b71-beed-f1d999dbd2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:05.576137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6890863-883d-4b71-beed-f1d999dbd2f0 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:05.578171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6890863-883d-4b71-beed-f1d999dbd2f0 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:05.580294 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6890863-883d-4b71-beed-f1d999dbd2f0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:05.582533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 150/287] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:05 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:05.583425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6890863-883d-4b71-beed-f1d999dbd2f0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:05.604234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:05.743810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6890863-883d-4b71-beed-f1d999dbd2f0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:05.750100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483d8ad-e8c0-4afe-adb6-482a86d9f0f6 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:05.800818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624571f-9a59-4260-b186-2ee441a6b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:05.802369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624571f-9a59-4260-b186-2ee441a6b59f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:05.803476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624571f-9a59-4260-b186-2ee441a6b59f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:05.803946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624571f-9a59-4260-b186-2ee441a6b59f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:05.805380 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624571f-9a59-4260-b186-2ee441a6b59f None admin] RBAC: Adding query filter params (role.id=44cbd523412b44e3a591b3111fb50a71, user.id=91580c57f0ee458c8aded161c84b7392, scope.system=all) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:05.805888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624571f-9a59-4260-b186-2ee441a6b59f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:05.807656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624571f-9a59-4260-b186-2ee441a6b59f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:05.811561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48527a4e-5c01-43c4-99a1-3b4d073ff727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:05.812877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48527a4e-5c01-43c4-99a1-3b4d073ff727 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:05.813454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48527a4e-5c01-43c4-99a1-3b4d073ff727 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:05.813884 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48527a4e-5c01-43c4-99a1-3b4d073ff727 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:05.814988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 138/288] 10.222.0.64 () {56 vars in 1308 bytes} [Mon Jun 27 09:15:05 2022] GET /identity/v3/role_assignments?user.id=91580c57f0ee458c8aded161c84b7392&scope.system=all&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:05.815235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48527a4e-5c01-43c4-99a1-3b4d073ff727 None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:05.815682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48527a4e-5c01-43c4-99a1-3b4d073ff727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:05.816542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624571f-9a59-4260-b186-2ee441a6b59f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:05.818512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48527a4e-5c01-43c4-99a1-3b4d073ff727 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:05.839646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 151/289] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:05.839646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48527a4e-5c01-43c4-99a1-3b4d073ff727 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:05.915653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:05.975665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624571f-9a59-4260-b186-2ee441a6b59f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:06.031477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5053a-e3a3-41ea-b696-797c4a787c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:06.033144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5053a-e3a3-41ea-b696-797c4a787c9b None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:06.033801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5053a-e3a3-41ea-b696-797c4a787c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:06.034268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5053a-e3a3-41ea-b696-797c4a787c9b None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126/users/590e237b883045aa84ae9823fa39391f/roles/536229a47a9d4a03b6d0b1c59fd8f39a` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:06.037262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5053a-e3a3-41ea-b696-797c4a787c9b None admin] RBAC: Authorizing `identity:create_grant(project_id=04cc0f746b104684b7b6e33d52a38126, user_id=590e237b883045aa84ae9823fa39391f, role_id=536229a47a9d4a03b6d0b1c59fd8f39a)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:06.039554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5053a-e3a3-41ea-b696-797c4a787c9b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:06.083718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:06.142512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 139/290] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:15:05 2022] PUT /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126/users/590e237b883045aa84ae9823fa39391f/roles/536229a47a9d4a03b6d0b1c59fd8f39a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:06.150611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48527a4e-5c01-43c4-99a1-3b4d073ff727 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:06.226790 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5a63a-4577-46a0-b649-fac41f96c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:06.228745 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5a63a-4577-46a0-b649-fac41f96c0ab None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:06.229783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5a63a-4577-46a0-b649-fac41f96c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:06.230624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5a63a-4577-46a0-b649-fac41f96c0ab None admin] PATH_INFO: `/v3/users/6d0b9295aa42422184417453cd03267f` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:06.233549 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5a63a-4577-46a0-b649-fac41f96c0ab None admin] RBAC: Authorizing `identity:get_user(user_id=6d0b9295aa42422184417453cd03267f)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:06.236947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5a63a-4577-46a0-b649-fac41f96c0ab None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:06.241997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 152/291] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:05 2022] GET /identity/v3/users/6d0b9295aa42422184417453cd03267f => generated 297 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:06.248917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5053a-e3a3-41ea-b696-797c4a787c9b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:06.469409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5053a-e3a3-41ea-b696-797c4a787c9b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:06.495102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:06.535548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a2e42-0552-40c2-8583-66a3600672ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:06.537354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a2e42-0552-40c2-8583-66a3600672ae None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:06.537841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a2e42-0552-40c2-8583-66a3600672ae None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:06.538327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a2e42-0552-40c2-8583-66a3600672ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:06.685048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a2e42-0552-40c2-8583-66a3600672ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:06.687858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a2e42-0552-40c2-8583-66a3600672ae None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:06.889614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 140/292] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:06.901585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5a63a-4577-46a0-b649-fac41f96c0ab None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:07.188433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5a63a-4577-46a0-b649-fac41f96c0ab None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:07.275582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053b6e1-df3d-45f4-9788-0ba5557a6a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:07.276844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053b6e1-df3d-45f4-9788-0ba5557a6a97 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:07.276844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053b6e1-df3d-45f4-9788-0ba5557a6a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:07.276844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053b6e1-df3d-45f4-9788-0ba5557a6a97 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:07.285120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053b6e1-df3d-45f4-9788-0ba5557a6a97 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:07.289381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053b6e1-df3d-45f4-9788-0ba5557a6a97 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:07.292133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 153/293] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:06 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:07.303583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a2e42-0552-40c2-8583-66a3600672ae None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:07.357895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:07.490550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a2e42-0552-40c2-8583-66a3600672ae None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:07.545243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaf1f8-6a41-47a3-b56d-b9f8e13e6295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:07.545997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaf1f8-6a41-47a3-b56d-b9f8e13e6295 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:07.546339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaf1f8-6a41-47a3-b56d-b9f8e13e6295 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:07.546623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaf1f8-6a41-47a3-b56d-b9f8e13e6295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:07.547504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beaf1f8-6a41-47a3-b56d-b9f8e13e6295 None admin] RBAC: Adding query filter params (role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=04cc0f746b104684b7b6e33d52a38126, user.id=6d0b9295aa42422184417453cd03267f) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:07.547851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beaf1f8-6a41-47a3-b56d-b9f8e13e6295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:07.549334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beaf1f8-6a41-47a3-b56d-b9f8e13e6295 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:07.561719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 141/294] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:07 2022] GET /identity/v3/role_assignments?user.id=6d0b9295aa42422184417453cd03267f&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:07.821721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305098cb-d44e-4ebe-b161-93606260ee07 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:07.822285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305098cb-d44e-4ebe-b161-93606260ee07 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:07.822731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305098cb-d44e-4ebe-b161-93606260ee07 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:07.824890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 154/295] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:07.832534 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec62a2c-cea9-4ab6-88f0-c89d089ebd18 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:07.833240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec62a2c-cea9-4ab6-88f0-c89d089ebd18 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:07.833728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec62a2c-cea9-4ab6-88f0-c89d089ebd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:07.854878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-bec62a2c-cea9-4ab6-88f0-c89d089ebd18 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:07.856259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-bec62a2c-cea9-4ab6-88f0-c89d089ebd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:07.954457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 142/296] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:07.978478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305098cb-d44e-4ebe-b161-93606260ee07 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:08.003305 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:08.125417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305098cb-d44e-4ebe-b161-93606260ee07 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:08.171706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310055c-16ca-4c9a-b756-035407124ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:08.173714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310055c-16ca-4c9a-b756-035407124ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:08.174524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310055c-16ca-4c9a-b756-035407124ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:08.175295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310055c-16ca-4c9a-b756-035407124ec1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:08.178733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310055c-16ca-4c9a-b756-035407124ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:08.183468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310055c-16ca-4c9a-b756-035407124ec1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:08.186421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 155/297] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:08.196045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec62a2c-cea9-4ab6-88f0-c89d089ebd18 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:08.225020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:08.368813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397c297-2b0e-486c-8894-5eb50adb75ea None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:08.369130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397c297-2b0e-486c-8894-5eb50adb75ea None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:08.369374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397c297-2b0e-486c-8894-5eb50adb75ea None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:08.372049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 156/298] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:08.378403 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4ead-ef10-4d47-87c1-8420107888b3 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:08.379014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4ead-ef10-4d47-87c1-8420107888b3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:08.379588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4ead-ef10-4d47-87c1-8420107888b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:08.401669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-098c4ead-ef10-4d47-87c1-8420107888b3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:08.403763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-098c4ead-ef10-4d47-87c1-8420107888b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:08.412180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec62a2c-cea9-4ab6-88f0-c89d089ebd18 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:08.472432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f0a86-63f7-4cbe-b4da-67a242bfd022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:08.473131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f0a86-63f7-4cbe-b4da-67a242bfd022 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:08.473996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f0a86-63f7-4cbe-b4da-67a242bfd022 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:08.474290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f0a86-63f7-4cbe-b4da-67a242bfd022 None admin] PATH_INFO: `/v3/groups` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:08.475010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f0a86-63f7-4cbe-b4da-67a242bfd022 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:08.476890 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f0a86-63f7-4cbe-b4da-67a242bfd022 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:08.524220 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 157/299] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:15:08.526885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc042a7e-ae6b-44fe-be84-0f86c58376f5 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:08.527457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc042a7e-ae6b-44fe-be84-0f86c58376f5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:08.527940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc042a7e-ae6b-44fe-be84-0f86c58376f5 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:08.530573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 158/300] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:08.545927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b063170-298b-4f98-b889-64ceb6383a9e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:08.546448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b063170-298b-4f98-b889-64ceb6383a9e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:08.546937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b063170-298b-4f98-b889-64ceb6383a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:08.575967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5b063170-298b-4f98-b889-64ceb6383a9e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:08.578036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5b063170-298b-4f98-b889-64ceb6383a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:08.582045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:08.615576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 143/301] 10.222.0.64 () {60 vars in 1134 bytes} [Mon Jun 27 09:15:08 2022] POST /identity/v3/groups => generated 229 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:15:08.615576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f0a86-63f7-4cbe-b4da-67a242bfd022 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:08.718860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 159/302] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:15:08.762037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b063170-298b-4f98-b889-64ceb6383a9e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:08.789336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:08.854845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f0a86-63f7-4cbe-b4da-67a242bfd022 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:08.930162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029fc22-1699-47c2-b1d5-bca1189fbfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:08.932147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029fc22-1699-47c2-b1d5-bca1189fbfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:08.932886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029fc22-1699-47c2-b1d5-bca1189fbfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:08.933533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029fc22-1699-47c2-b1d5-bca1189fbfa8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:08.952279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029fc22-1699-47c2-b1d5-bca1189fbfa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-9029fc22-1699-47c2-b1d5-bca1189fbfa8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:08.966511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:08.968076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:08.969271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 144/303] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:08.971181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae21d5-ad6f-4a00-9909-7e67417f9810 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:08.971678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae21d5-ad6f-4a00-9909-7e67417f9810 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:08.972012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae21d5-ad6f-4a00-9909-7e67417f9810 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:08.974860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 145/304] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:08.977492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b063170-298b-4f98-b889-64ceb6383a9e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:08.984872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a42067-af1f-4f98-8768-124e037dfefe None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:08.985354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a42067-af1f-4f98-8768-124e037dfefe None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:08.985905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a42067-af1f-4f98-8768-124e037dfefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:09.015120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b7a42067-af1f-4f98-8768-124e037dfefe None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:09.017211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b7a42067-af1f-4f98-8768-124e037dfefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:09.036651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86957836-f348-4da6-9657-bd0290849c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:09.037958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86957836-f348-4da6-9657-bd0290849c2a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:09.038414 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86957836-f348-4da6-9657-bd0290849c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:09.038836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86957836-f348-4da6-9657-bd0290849c2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:09.051565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:09.058537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86957836-f348-4da6-9657-bd0290849c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-86957836-f348-4da6-9657-bd0290849c2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:09.072676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:09.074789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 160/305] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:09.077227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86957836-f348-4da6-9657-bd0290849c2a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:09.140077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 146/306] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:09.142141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a42067-af1f-4f98-8768-124e037dfefe None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:09.250174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:09.278869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86957836-f348-4da6-9657-bd0290849c2a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:09.326964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a42067-af1f-4f98-8768-124e037dfefe None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:09.340349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2e4e2-1e95-48a7-81c2-416a79b447f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:09.341217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2e4e2-1e95-48a7-81c2-416a79b447f0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:09.342419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2e4e2-1e95-48a7-81c2-416a79b447f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:09.342869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2e4e2-1e95-48a7-81c2-416a79b447f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:09.343853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2e4e2-1e95-48a7-81c2-416a79b447f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:09.344293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2e4e2-1e95-48a7-81c2-416a79b447f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:09.346527 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2e4e2-1e95-48a7-81c2-416a79b447f0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:09.359070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 161/307] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:09.365754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2e4e2-1e95-48a7-81c2-416a79b447f0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:09.396292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8836a1-e6cc-4d06-ac7f-4b14e7c15b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:09.397460 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8836a1-e6cc-4d06-ac7f-4b14e7c15b2f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:09.397966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8836a1-e6cc-4d06-ac7f-4b14e7c15b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:09.398448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8836a1-e6cc-4d06-ac7f-4b14e7c15b2f None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:09.400218 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8836a1-e6cc-4d06-ac7f-4b14e7c15b2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:09.400784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8836a1-e6cc-4d06-ac7f-4b14e7c15b2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:09.403492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8836a1-e6cc-4d06-ac7f-4b14e7c15b2f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:09.420553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 147/308] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:09.422156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8836a1-e6cc-4d06-ac7f-4b14e7c15b2f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:09.467168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:09.548404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2e4e2-1e95-48a7-81c2-416a79b447f0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:09.594963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa2339-8522-41aa-9d44-d4def9435097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:09.596024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa2339-8522-41aa-9d44-d4def9435097 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:09.596493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa2339-8522-41aa-9d44-d4def9435097 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:09.596959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa2339-8522-41aa-9d44-d4def9435097 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:09.607722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8836a1-e6cc-4d06-ac7f-4b14e7c15b2f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:09.613531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa2339-8522-41aa-9d44-d4def9435097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-42fa2339-8522-41aa-9d44-d4def9435097 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:09.623783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:09.625558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 162/309] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:09.628394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa2339-8522-41aa-9d44-d4def9435097 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:09.659822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:09.680051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675aa4f-38e0-4954-ac4f-b056b5e84f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:09.681551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675aa4f-38e0-4954-ac4f-b056b5e84f85 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:09.682033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675aa4f-38e0-4954-ac4f-b056b5e84f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:09.682519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675aa4f-38e0-4954-ac4f-b056b5e84f85 None admin] PATH_INFO: `/v3/users/91580c57f0ee458c8aded161c84b7392` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:09.684835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675aa4f-38e0-4954-ac4f-b056b5e84f85 None admin] RBAC: Authorizing `identity:get_user(user_id=91580c57f0ee458c8aded161c84b7392)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:09.687631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675aa4f-38e0-4954-ac4f-b056b5e84f85 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:09.690995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 148/310] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:09 2022] GET /identity/v3/users/91580c57f0ee458c8aded161c84b7392 => generated 293 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:09.692854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675aa4f-38e0-4954-ac4f-b056b5e84f85 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:09.821444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa2339-8522-41aa-9d44-d4def9435097 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:09.862337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:09.875502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675aa4f-38e0-4954-ac4f-b056b5e84f85 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:09.892622 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9df7d5-246b-4f3f-81ae-30aebc5d2884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:09.894580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9df7d5-246b-4f3f-81ae-30aebc5d2884 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:09.895343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9df7d5-246b-4f3f-81ae-30aebc5d2884 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:09.895883 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9df7d5-246b-4f3f-81ae-30aebc5d2884 None admin] PATH_INFO: `/v3/users/6a53d8205fb64099901af9469b01f1a5` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:09.921679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae65ed7-ee2b-455c-be99-70d4d010406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:09.922182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae65ed7-ee2b-455c-be99-70d4d010406f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:09.922524 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae65ed7-ee2b-455c-be99-70d4d010406f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:09.922758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae65ed7-ee2b-455c-be99-70d4d010406f None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:09.923641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae65ed7-ee2b-455c-be99-70d4d010406f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:09.923905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae65ed7-ee2b-455c-be99-70d4d010406f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:09.924071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9df7d5-246b-4f3f-81ae-30aebc5d2884 None admin] RBAC: Authorizing `identity:get_user(user_id=6a53d8205fb64099901af9469b01f1a5)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:09.925297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae65ed7-ee2b-455c-be99-70d4d010406f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:09.927131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9df7d5-246b-4f3f-81ae-30aebc5d2884 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:09.930647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 163/311] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:09 2022] GET /identity/v3/users/6a53d8205fb64099901af9469b01f1a5 => generated 293 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:09.931809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9df7d5-246b-4f3f-81ae-30aebc5d2884 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:09.948301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 149/312] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:09.951084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae65ed7-ee2b-455c-be99-70d4d010406f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:10.046864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:10.123096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9df7d5-246b-4f3f-81ae-30aebc5d2884 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:10.137111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae65ed7-ee2b-455c-be99-70d4d010406f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:10.186935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e510ddb-b050-47de-bde9-05a38900d93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:10.188102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e510ddb-b050-47de-bde9-05a38900d93a None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:10.188548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e510ddb-b050-47de-bde9-05a38900d93a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:10.188990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e510ddb-b050-47de-bde9-05a38900d93a None admin] PATH_INFO: `/v3/system/users/91580c57f0ee458c8aded161c84b7392/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:10.191620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e510ddb-b050-47de-bde9-05a38900d93a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=91580c57f0ee458c8aded161c84b7392, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:10.193655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e510ddb-b050-47de-bde9-05a38900d93a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:10.201763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5dc77-f5a3-4221-a248-50b118236540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:10.202586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5dc77-f5a3-4221-a248-50b118236540 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:10.202909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5dc77-f5a3-4221-a248-50b118236540 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:10.203220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5dc77-f5a3-4221-a248-50b118236540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:10.204362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5dc77-f5a3-4221-a248-50b118236540 None admin] RBAC: Adding query filter params (role.id=536229a47a9d4a03b6d0b1c59fd8f39a, user.id=6a53d8205fb64099901af9469b01f1a5, scope.system=all) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:10.204750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5dc77-f5a3-4221-a248-50b118236540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:10.206521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5dc77-f5a3-4221-a248-50b118236540 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:10.213776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 150/313] 10.222.0.64 () {56 vars in 1308 bytes} [Mon Jun 27 09:15:09 2022] GET /identity/v3/role_assignments?user.id=6a53d8205fb64099901af9469b01f1a5&scope.system=all&role.id=536229a47a9d4a03b6d0b1c59fd8f39a => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:10.215891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5dc77-f5a3-4221-a248-50b118236540 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:10.288732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 164/314] 10.222.0.64 () {58 vars in 1334 bytes} [Mon Jun 27 09:15:09 2022] PUT /identity/v3/system/users/91580c57f0ee458c8aded161c84b7392/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:10.389054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5dc77-f5a3-4221-a248-50b118236540 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:10.413755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:10.436437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed350-3359-4850-b570-ca9d31abd271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:10.437387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed350-3359-4850-b570-ca9d31abd271 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:10.437702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed350-3359-4850-b570-ca9d31abd271 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:10.437978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ed350-3359-4850-b570-ca9d31abd271 None admin] PATH_INFO: `/v3/users/590e237b883045aa84ae9823fa39391f` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:10.439652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ed350-3359-4850-b570-ca9d31abd271 None admin] RBAC: Authorizing `identity:get_user(user_id=590e237b883045aa84ae9823fa39391f)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:10.441209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ed350-3359-4850-b570-ca9d31abd271 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:10.444044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 151/315] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:10 2022] GET /identity/v3/users/590e237b883045aa84ae9823fa39391f => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:10.454165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e510ddb-b050-47de-bde9-05a38900d93a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:10.554726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:10.680462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e510ddb-b050-47de-bde9-05a38900d93a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:10.751583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3feb66-81b3-493b-a75d-326a62e1db32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:10.753051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3feb66-81b3-493b-a75d-326a62e1db32 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:10.753803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3feb66-81b3-493b-a75d-326a62e1db32 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:10.755083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3feb66-81b3-493b-a75d-326a62e1db32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:10.870546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3feb66-81b3-493b-a75d-326a62e1db32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:10.872101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3feb66-81b3-493b-a75d-326a62e1db32 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:10.934319 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:10.992021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 165/316] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 09:15:11.000542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed350-3359-4850-b570-ca9d31abd271 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:11.117912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fc19e-e32e-4e95-a141-139bfffb5d25 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:11.118252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fc19e-e32e-4e95-a141-139bfffb5d25 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:11.118465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fc19e-e32e-4e95-a141-139bfffb5d25 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:11.120118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 166/317] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:11.126184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f217556-75ce-4c72-a28f-a93968d9ec77 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:11.126959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f217556-75ce-4c72-a28f-a93968d9ec77 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:11.126959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f217556-75ce-4c72-a28f-a93968d9ec77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:11.132473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ed350-3359-4850-b570-ca9d31abd271 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:11.150265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-0f217556-75ce-4c72-a28f-a93968d9ec77 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:11.152112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-0f217556-75ce-4c72-a28f-a93968d9ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:11.177391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e7aeb-fe01-4b58-be7a-de24b5a07ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:11.178756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e7aeb-fe01-4b58-be7a-de24b5a07ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:11.179275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e7aeb-fe01-4b58-be7a-de24b5a07ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:11.179775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e7aeb-fe01-4b58-be7a-de24b5a07ab7 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:11.182513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e7aeb-fe01-4b58-be7a-de24b5a07ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:11.184688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e7aeb-fe01-4b58-be7a-de24b5a07ab7 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:11.187652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 152/318] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:10 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:11.197031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e7aeb-fe01-4b58-be7a-de24b5a07ab7 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:11.331274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 167/319] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:11.354948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f217556-75ce-4c72-a28f-a93968d9ec77 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:11.390889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:11.444872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:11.484291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e7aeb-fe01-4b58-be7a-de24b5a07ab7 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:11.502291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f217556-75ce-4c72-a28f-a93968d9ec77 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:11.554386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a14d9f-92b1-41fe-8f61-32eb4665d966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:11.556013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a14d9f-92b1-41fe-8f61-32eb4665d966 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:11.556833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a14d9f-92b1-41fe-8f61-32eb4665d966 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:11.557498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a14d9f-92b1-41fe-8f61-32eb4665d966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:11.559212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a14d9f-92b1-41fe-8f61-32eb4665d966 None admin] RBAC: Adding query filter params (role.id=536229a47a9d4a03b6d0b1c59fd8f39a, scope.project.id=04cc0f746b104684b7b6e33d52a38126, user.id=590e237b883045aa84ae9823fa39391f) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:11.560077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a14d9f-92b1-41fe-8f61-32eb4665d966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:11.563222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a14d9f-92b1-41fe-8f61-32eb4665d966 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:11.568823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa38122-a776-418d-8043-ee556b45f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:11.570331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38122-a776-418d-8043-ee556b45f1cc None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:11.570919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38122-a776-418d-8043-ee556b45f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:11.571450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38122-a776-418d-8043-ee556b45f1cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:11.573611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa38122-a776-418d-8043-ee556b45f1cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:11.576447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa38122-a776-418d-8043-ee556b45f1cc None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:11.576988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 153/320] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:11 2022] GET /identity/v3/role_assignments?user.id=590e237b883045aa84ae9823fa39391f&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=536229a47a9d4a03b6d0b1c59fd8f39a => generated 608 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:11.579691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 168/321] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:11.596727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a14d9f-92b1-41fe-8f61-32eb4665d966 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:11.789860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a14d9f-92b1-41fe-8f61-32eb4665d966 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:11.823962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:11.858654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f5957-aa7a-4ea9-b236-9556d03c119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:11.860170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f5957-aa7a-4ea9-b236-9556d03c119e None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:11.860662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f5957-aa7a-4ea9-b236-9556d03c119e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:11.861156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f5957-aa7a-4ea9-b236-9556d03c119e None admin] PATH_INFO: `/v3/groups` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:11.862552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f5957-aa7a-4ea9-b236-9556d03c119e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:11.865434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f5957-aa7a-4ea9-b236-9556d03c119e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:11.960518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 154/322] 10.222.0.64 () {60 vars in 1134 bytes} [Mon Jun 27 09:15:11 2022] POST /identity/v3/groups => generated 226 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:15:13.567105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534c694-560e-4869-8069-7bf31c6740ac None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:13.567858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534c694-560e-4869-8069-7bf31c6740ac None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:13.568190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534c694-560e-4869-8069-7bf31c6740ac None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:13.570675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 169/323] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:13.581037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44aa72a-2c70-4d5b-970a-92fb91615d75 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:13.581773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44aa72a-2c70-4d5b-970a-92fb91615d75 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:13.582079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44aa72a-2c70-4d5b-970a-92fb91615d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:13.615720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b44aa72a-2c70-4d5b-970a-92fb91615d75 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:13.617820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b44aa72a-2c70-4d5b-970a-92fb91615d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:13.672038 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da87b6-72d8-412b-b3dd-3879b699318e None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:13.672516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da87b6-72d8-412b-b3dd-3879b699318e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:13.672962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da87b6-72d8-412b-b3dd-3879b699318e None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:13.675561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 170/324] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:15:13.684548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c231-cbcc-4e21-a4ca-fc8cc8905ebc None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:13.685003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c231-cbcc-4e21-a4ca-fc8cc8905ebc None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:13.685442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c231-cbcc-4e21-a4ca-fc8cc8905ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:13.744771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2445c231-cbcc-4e21-a4ca-fc8cc8905ebc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:13.746034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2445c231-cbcc-4e21-a4ca-fc8cc8905ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:13.792570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 155/325] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:15:13.860790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44aa72a-2c70-4d5b-970a-92fb91615d75 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:13.877322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 171/326] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:15:13.912715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c231-cbcc-4e21-a4ca-fc8cc8905ebc None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:14.070767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:14.112231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c231-cbcc-4e21-a4ca-fc8cc8905ebc None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:14.158969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44aa72a-2c70-4d5b-970a-92fb91615d75 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:14.174408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc704b24-b1b5-42cc-b08b-ea39b237686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:14.175247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc704b24-b1b5-42cc-b08b-ea39b237686c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:14.179568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc704b24-b1b5-42cc-b08b-ea39b237686c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:14.179568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc704b24-b1b5-42cc-b08b-ea39b237686c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:14.214197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc704b24-b1b5-42cc-b08b-ea39b237686c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-bc704b24-b1b5-42cc-b08b-ea39b237686c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:14.214300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:14.215385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:14.216744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 172/327] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:15:14.224655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc704b24-b1b5-42cc-b08b-ea39b237686c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:14.293923 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ecf70-ceea-4b60-8592-d6a99d8b84b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:14.295850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ecf70-ceea-4b60-8592-d6a99d8b84b2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:14.296667 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ecf70-ceea-4b60-8592-d6a99d8b84b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:14.297422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ecf70-ceea-4b60-8592-d6a99d8b84b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:14.341304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ecf70-ceea-4b60-8592-d6a99d8b84b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-5a9ecf70-ceea-4b60-8592-d6a99d8b84b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:14.374191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:14.375983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:14.377619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:14.382685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 156/328] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 523 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:15:14.392833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ecf70-ceea-4b60-8592-d6a99d8b84b2 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:14.393281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc704b24-b1b5-42cc-b08b-ea39b237686c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:14.443153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:14.457633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43c44e-6aa7-43ef-b30c-0e19d0aa34ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:14.458841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43c44e-6aa7-43ef-b30c-0e19d0aa34ba None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:14.459128 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43c44e-6aa7-43ef-b30c-0e19d0aa34ba None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:14.459406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43c44e-6aa7-43ef-b30c-0e19d0aa34ba None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:14.460216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43c44e-6aa7-43ef-b30c-0e19d0aa34ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:14.460526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43c44e-6aa7-43ef-b30c-0e19d0aa34ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:14.461956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43c44e-6aa7-43ef-b30c-0e19d0aa34ba None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:14.477104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 173/329] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:14.482659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43c44e-6aa7-43ef-b30c-0e19d0aa34ba None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:14.710696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43c44e-6aa7-43ef-b30c-0e19d0aa34ba None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:14.716181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ecf70-ceea-4b60-8592-d6a99d8b84b2 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:14.770445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05bebd-e68c-4be8-b7a8-97b3db533f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:14.772023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05bebd-e68c-4be8-b7a8-97b3db533f9c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:14.772513 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05bebd-e68c-4be8-b7a8-97b3db533f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:14.772998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05bebd-e68c-4be8-b7a8-97b3db533f9c None admin] PATH_INFO: `/v3/users/6a53d8205fb64099901af9469b01f1a5` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:14.778996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05bebd-e68c-4be8-b7a8-97b3db533f9c None admin] RBAC: Authorizing `identity:get_user(user_id=6a53d8205fb64099901af9469b01f1a5)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:14.785753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05bebd-e68c-4be8-b7a8-97b3db533f9c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:14.788862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 174/330] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:14 2022] GET /identity/v3/users/6a53d8205fb64099901af9469b01f1a5 => generated 293 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:14.796542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05bebd-e68c-4be8-b7a8-97b3db533f9c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:14.855506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a32fe2-0ccf-495d-976a-36d8f7d2795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:14.859098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a32fe2-0ccf-495d-976a-36d8f7d2795d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:14.859098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a32fe2-0ccf-495d-976a-36d8f7d2795d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:14.859098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a32fe2-0ccf-495d-976a-36d8f7d2795d None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:14.859098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a32fe2-0ccf-495d-976a-36d8f7d2795d None admin] RBAC: Adding query filter params (name=member) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:14.859667 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a32fe2-0ccf-495d-976a-36d8f7d2795d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:14.862257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a32fe2-0ccf-495d-976a-36d8f7d2795d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:14.888332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 157/331] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:14.900100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a32fe2-0ccf-495d-976a-36d8f7d2795d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:14.964633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:15.025058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:15.055225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05bebd-e68c-4be8-b7a8-97b3db533f9c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:15.134383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdc53d-5dfa-4c63-a16b-462de6f4495c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:15.135841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdc53d-5dfa-4c63-a16b-462de6f4495c None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:15.136391 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdc53d-5dfa-4c63-a16b-462de6f4495c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:15.136909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdc53d-5dfa-4c63-a16b-462de6f4495c None admin] PATH_INFO: `/v3/system/users/6a53d8205fb64099901af9469b01f1a5/roles/536229a47a9d4a03b6d0b1c59fd8f39a` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:15.141689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdc53d-5dfa-4c63-a16b-462de6f4495c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6a53d8205fb64099901af9469b01f1a5, role_id=536229a47a9d4a03b6d0b1c59fd8f39a)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:15.143551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdc53d-5dfa-4c63-a16b-462de6f4495c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:15.221938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 175/332] 10.222.0.64 () {58 vars in 1334 bytes} [Mon Jun 27 09:15:14 2022] PUT /identity/v3/system/users/6a53d8205fb64099901af9469b01f1a5/roles/536229a47a9d4a03b6d0b1c59fd8f39a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:15.244706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a32fe2-0ccf-495d-976a-36d8f7d2795d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:15.275475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdea21-84b3-45bc-98ad-bf5ad4450134 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:15.277221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdea21-84b3-45bc-98ad-bf5ad4450134 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:15.277573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdea21-84b3-45bc-98ad-bf5ad4450134 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:15.281331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 176/333] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:15.287712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0329c4-326d-418e-8af1-aafdc6ddae30 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:15.288175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0329c4-326d-418e-8af1-aafdc6ddae30 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:15.288651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0329c4-326d-418e-8af1-aafdc6ddae30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:15.332619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-1c0329c4-326d-418e-8af1-aafdc6ddae30 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:15.334353 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1c0329c4-326d-418e-8af1-aafdc6ddae30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:15.372700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997a74a-975c-4aab-9f95-dc5553e067d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:15.374452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997a74a-975c-4aab-9f95-dc5553e067d3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:15.375138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997a74a-975c-4aab-9f95-dc5553e067d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:15.375828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997a74a-975c-4aab-9f95-dc5553e067d3 None admin] PATH_INFO: `/v3/users/91580c57f0ee458c8aded161c84b7392` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:15.378221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997a74a-975c-4aab-9f95-dc5553e067d3 None admin] RBAC: Authorizing `identity:get_user(user_id=91580c57f0ee458c8aded161c84b7392)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:15.381699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997a74a-975c-4aab-9f95-dc5553e067d3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:15.385256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 158/334] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:14 2022] GET /identity/v3/users/91580c57f0ee458c8aded161c84b7392 => generated 293 bytes in 487 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:15.418824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997a74a-975c-4aab-9f95-dc5553e067d3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:15.448860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 177/335] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 27 09:15:15.565904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0329c4-326d-418e-8af1-aafdc6ddae30 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:15.667939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:15.679566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:15.684416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997a74a-975c-4aab-9f95-dc5553e067d3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:15.739584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0329c4-326d-418e-8af1-aafdc6ddae30 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:15.749542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13e788-b254-4272-9401-29e27737806e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:15.750992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13e788-b254-4272-9401-29e27737806e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:15.751488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13e788-b254-4272-9401-29e27737806e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:15.752033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13e788-b254-4272-9401-29e27737806e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:15.753116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13e788-b254-4272-9401-29e27737806e None admin] RBAC: Adding query filter params (role.id=44cbd523412b44e3a591b3111fb50a71, user.id=91580c57f0ee458c8aded161c84b7392, scope.system=all) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:15.753656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13e788-b254-4272-9401-29e27737806e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:15.755466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13e788-b254-4272-9401-29e27737806e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:15.779874 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 159/336] 10.222.0.64 () {56 vars in 1308 bytes} [Mon Jun 27 09:15:15 2022] GET /identity/v3/role_assignments?user.id=91580c57f0ee458c8aded161c84b7392&scope.system=all&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 510 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:15.793538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84848f0c-5141-4953-bb0c-e1bfd6333539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:15.794449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84848f0c-5141-4953-bb0c-e1bfd6333539 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:15.794738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84848f0c-5141-4953-bb0c-e1bfd6333539 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:15.795020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84848f0c-5141-4953-bb0c-e1bfd6333539 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:15.814989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84848f0c-5141-4953-bb0c-e1bfd6333539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-84848f0c-5141-4953-bb0c-e1bfd6333539 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:15.823909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:15.824902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:15.825772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:15.831676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 178/337] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:15.834388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13e788-b254-4272-9401-29e27737806e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:15.992713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13e788-b254-4272-9401-29e27737806e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:16.043320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8762e9-7e04-483f-b659-ecb5f824535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:16.044449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8762e9-7e04-483f-b659-ecb5f824535e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:16.044979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8762e9-7e04-483f-b659-ecb5f824535e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:16.045506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8762e9-7e04-483f-b659-ecb5f824535e None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:16.046582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8762e9-7e04-483f-b659-ecb5f824535e None admin] RBAC: Adding query filter params (name=member) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:16.047148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8762e9-7e04-483f-b659-ecb5f824535e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:16.048955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8762e9-7e04-483f-b659-ecb5f824535e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:16.064530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 160/338] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:16.072465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84848f0c-5141-4953-bb0c-e1bfd6333539 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:16.246510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84848f0c-5141-4953-bb0c-e1bfd6333539 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:16.266151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:16.299278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffb4fa-aaf1-445c-8fe1-45d1b0de4350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:16.300276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffb4fa-aaf1-445c-8fe1-45d1b0de4350 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:16.300566 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffb4fa-aaf1-445c-8fe1-45d1b0de4350 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:16.300849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffb4fa-aaf1-445c-8fe1-45d1b0de4350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:16.355789 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc24ba-f584-4341-adb6-f2cf3ad9da4e None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:16.356500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc24ba-f584-4341-adb6-f2cf3ad9da4e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:16.357004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc24ba-f584-4341-adb6-f2cf3ad9da4e None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:16.358755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 161/339] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:16.366998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82502566-32eb-4d96-9562-adf01d5bb1bb None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:16.367725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82502566-32eb-4d96-9562-adf01d5bb1bb None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:16.368229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82502566-32eb-4d96-9562-adf01d5bb1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:16.392662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-82502566-32eb-4d96-9562-adf01d5bb1bb None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:16.394278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-82502566-32eb-4d96-9562-adf01d5bb1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:16.401465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffb4fa-aaf1-445c-8fe1-45d1b0de4350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:16.401863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffb4fa-aaf1-445c-8fe1-45d1b0de4350 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:16.414513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:16.482082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 162/340] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:16.506807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82502566-32eb-4d96-9562-adf01d5bb1bb None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:16.566476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 179/341] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:16.575502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffb4fa-aaf1-445c-8fe1-45d1b0de4350 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:16.612752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:16.630266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82502566-32eb-4d96-9562-adf01d5bb1bb None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:16.672927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5b136-b50e-46a3-8c6d-cdcf281784c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:16.674042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5b136-b50e-46a3-8c6d-cdcf281784c1 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:16.674546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5b136-b50e-46a3-8c6d-cdcf281784c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:16.675024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5b136-b50e-46a3-8c6d-cdcf281784c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:16.688270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5b136-b50e-46a3-8c6d-cdcf281784c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-6ec5b136-b50e-46a3-8c6d-cdcf281784c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:16.699070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:16.700134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:16.700920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:16.703249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 163/342] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:15:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:15:16.710483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5b136-b50e-46a3-8c6d-cdcf281784c1 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:16.724876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:16.806030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffb4fa-aaf1-445c-8fe1-45d1b0de4350 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:16.851895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5b136-b50e-46a3-8c6d-cdcf281784c1 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:16.885254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32805cdc-87d9-4965-856d-640b06da3b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:16.886615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805cdc-87d9-4965-856d-640b06da3b05 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:16.887140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805cdc-87d9-4965-856d-640b06da3b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:16.887650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32805cdc-87d9-4965-856d-640b06da3b05 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:16.889582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32805cdc-87d9-4965-856d-640b06da3b05 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:16.891352 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32805cdc-87d9-4965-856d-640b06da3b05 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:16.894347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 180/343] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:16 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:16.898253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff85dd2-fcb6-46d3-bdee-db3f90e81480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:16.898920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff85dd2-fcb6-46d3-bdee-db3f90e81480 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:16.899229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff85dd2-fcb6-46d3-bdee-db3f90e81480 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:16.899503 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff85dd2-fcb6-46d3-bdee-db3f90e81480 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:16.900570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff85dd2-fcb6-46d3-bdee-db3f90e81480 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:16.900878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff85dd2-fcb6-46d3-bdee-db3f90e81480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:16.902423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff85dd2-fcb6-46d3-bdee-db3f90e81480 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:16.902865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32805cdc-87d9-4965-856d-640b06da3b05 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:16.917870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 164/344] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:16.919629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff85dd2-fcb6-46d3-bdee-db3f90e81480 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:17.014307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:17.027409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:17.040078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32805cdc-87d9-4965-856d-640b06da3b05 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:17.068875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff85dd2-fcb6-46d3-bdee-db3f90e81480 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:17.093550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a4cb6-ba0f-41d1-a680-9bd9c6d49678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:17.094621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a4cb6-ba0f-41d1-a680-9bd9c6d49678 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:17.094978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a4cb6-ba0f-41d1-a680-9bd9c6d49678 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:17.095960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a4cb6-ba0f-41d1-a680-9bd9c6d49678 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:17.104193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a4cb6-ba0f-41d1-a680-9bd9c6d49678 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:17.106264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a4cb6-ba0f-41d1-a680-9bd9c6d49678 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:17.108991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 181/345] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:16 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:17.117412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a4cb6-ba0f-41d1-a680-9bd9c6d49678 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:17.119150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7cfb0-3046-4c53-99e8-3581eb8ba3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:17.120180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7cfb0-3046-4c53-99e8-3581eb8ba3bb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:17.120791 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7cfb0-3046-4c53-99e8-3581eb8ba3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:17.121314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7cfb0-3046-4c53-99e8-3581eb8ba3bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:17.213190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7cfb0-3046-4c53-99e8-3581eb8ba3bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:17.215185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7cfb0-3046-4c53-99e8-3581eb8ba3bb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:17.293205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a4cb6-ba0f-41d1-a680-9bd9c6d49678 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:17.304032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:17.355915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4310ac-81c0-4e17-bb84-524df1c5f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:17.357220 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4310ac-81c0-4e17-bb84-524df1c5f37d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:17.357689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4310ac-81c0-4e17-bb84-524df1c5f37d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:17.358118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4310ac-81c0-4e17-bb84-524df1c5f37d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:17.359385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4310ac-81c0-4e17-bb84-524df1c5f37d None admin] RBAC: Adding query filter params (group.id=129eb944ca1047a1ac4820bd64d84ae8, role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=8c495a22d3f7484cb377da68dab0bf97) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:17.360033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4310ac-81c0-4e17-bb84-524df1c5f37d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:17.361847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4310ac-81c0-4e17-bb84-524df1c5f37d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:17.369557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 182/346] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:17 2022] GET /identity/v3/role_assignments?group.id=129eb944ca1047a1ac4820bd64d84ae8&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:17.369744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 165/347] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 09:15:17.377808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7cfb0-3046-4c53-99e8-3581eb8ba3bb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:17.534387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7cfb0-3046-4c53-99e8-3581eb8ba3bb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:17.580889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51ba47-e803-44c1-bb67-e4250849d623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:17.581816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51ba47-e803-44c1-bb67-e4250849d623 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:17.582100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51ba47-e803-44c1-bb67-e4250849d623 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:17.582408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51ba47-e803-44c1-bb67-e4250849d623 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:17.583802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51ba47-e803-44c1-bb67-e4250849d623 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:17.585285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51ba47-e803-44c1-bb67-e4250849d623 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:17.587570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 166/348] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:17 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:17.593487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4310ac-81c0-4e17-bb84-524df1c5f37d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:17.608450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:17.735283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4310ac-81c0-4e17-bb84-524df1c5f37d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:17.785754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682590f8-fdcb-4b5e-8ead-251bbc838327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:17.786687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682590f8-fdcb-4b5e-8ead-251bbc838327 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:17.786973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682590f8-fdcb-4b5e-8ead-251bbc838327 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:17.787251 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682590f8-fdcb-4b5e-8ead-251bbc838327 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:17.792535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682590f8-fdcb-4b5e-8ead-251bbc838327 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:17.794182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682590f8-fdcb-4b5e-8ead-251bbc838327 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:17.796329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 183/349] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:17 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:17.803071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51ba47-e803-44c1-bb67-e4250849d623 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:17.868075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:17.961234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51ba47-e803-44c1-bb67-e4250849d623 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:18.012367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32751fee-dc19-475d-af81-130d70dc17b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:18.013567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751fee-dc19-475d-af81-130d70dc17b3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:18.014073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751fee-dc19-475d-af81-130d70dc17b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:18.014578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751fee-dc19-475d-af81-130d70dc17b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:18.015768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32751fee-dc19-475d-af81-130d70dc17b3 None admin] RBAC: Adding query filter params (group.id=129eb944ca1047a1ac4820bd64d84ae8, role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=8c495a22d3f7484cb377da68dab0bf97) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:18.016322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32751fee-dc19-475d-af81-130d70dc17b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:18.018013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32751fee-dc19-475d-af81-130d70dc17b3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:18.028787 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 167/350] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:17 2022] GET /identity/v3/role_assignments?group.id=129eb944ca1047a1ac4820bd64d84ae8&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:18.488657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1736b43-2434-4ab1-a59b-decc55d280f0 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:18.489178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1736b43-2434-4ab1-a59b-decc55d280f0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:18.489252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1736b43-2434-4ab1-a59b-decc55d280f0 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:18.490698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 184/351] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:15:18.501704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce42da-460d-4b3e-a2ed-b40a61d86b84 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:18.502321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce42da-460d-4b3e-a2ed-b40a61d86b84 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:18.502776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce42da-460d-4b3e-a2ed-b40a61d86b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:18.524013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-47ce42da-460d-4b3e-a2ed-b40a61d86b84 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:18.525525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-47ce42da-460d-4b3e-a2ed-b40a61d86b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:18.622446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 168/352] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:18.650642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1736b43-2434-4ab1-a59b-decc55d280f0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:18.777346 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:18.804446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1736b43-2434-4ab1-a59b-decc55d280f0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:18.829804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b6331-122c-4b87-a30c-a36b4a90f76f None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:18.830194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b6331-122c-4b87-a30c-a36b4a90f76f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:18.830367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b6331-122c-4b87-a30c-a36b4a90f76f None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:18.835582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 169/353] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:18.840052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850b661-5a8e-452a-a6e2-2c9bd9902d43 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:18.840630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850b661-5a8e-452a-a6e2-2c9bd9902d43 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:18.841712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850b661-5a8e-452a-a6e2-2c9bd9902d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:18.873882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-3850b661-5a8e-452a-a6e2-2c9bd9902d43 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:18.875911 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3850b661-5a8e-452a-a6e2-2c9bd9902d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:18.884487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf999b-ad1a-4264-b65e-4c14a542570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:18.886794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf999b-ad1a-4264-b65e-4c14a542570f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:18.887280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf999b-ad1a-4264-b65e-4c14a542570f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:18.887773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf999b-ad1a-4264-b65e-4c14a542570f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:18.916032 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf999b-ad1a-4264-b65e-4c14a542570f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-fcbf999b-ad1a-4264-b65e-4c14a542570f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:18.931264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:18.932446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:18.933454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:18.935865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 185/354] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:15:18.936500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:18.944706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf999b-ad1a-4264-b65e-4c14a542570f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:19.022010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 170/355] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:15:19.105996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850b661-5a8e-452a-a6e2-2c9bd9902d43 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:19.146884 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf999b-ad1a-4264-b65e-4c14a542570f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:19.226641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4395b0d-6b7d-4d6d-93d5-aa8ac8c21ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:19.227682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4395b0d-6b7d-4d6d-93d5-aa8ac8c21ecc None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:19.228148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4395b0d-6b7d-4d6d-93d5-aa8ac8c21ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:19.228629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4395b0d-6b7d-4d6d-93d5-aa8ac8c21ecc None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:19.229600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4395b0d-6b7d-4d6d-93d5-aa8ac8c21ecc None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:19.230111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4395b0d-6b7d-4d6d-93d5-aa8ac8c21ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:19.231787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4395b0d-6b7d-4d6d-93d5-aa8ac8c21ecc None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:19.248123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:19.250863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 186/356] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:19.257803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4395b0d-6b7d-4d6d-93d5-aa8ac8c21ecc None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:19.340588 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850b661-5a8e-452a-a6e2-2c9bd9902d43 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:19.412452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c79cd-7d8f-4285-b1f2-7ca475d2c93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:19.413206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c79cd-7d8f-4285-b1f2-7ca475d2c93e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:19.413599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c79cd-7d8f-4285-b1f2-7ca475d2c93e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:19.413880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c79cd-7d8f-4285-b1f2-7ca475d2c93e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:19.436551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c79cd-7d8f-4285-b1f2-7ca475d2c93e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-c26c79cd-7d8f-4285-b1f2-7ca475d2c93e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:19.446919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:19.452131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:19.453042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:19.456962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 171/357] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:15:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:19.465837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c79cd-7d8f-4285-b1f2-7ca475d2c93e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:19.511391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:19.539217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4395b0d-6b7d-4d6d-93d5-aa8ac8c21ecc None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:19.650096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79863f87-834f-4c59-aa15-55311477967e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:19.651428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79863f87-834f-4c59-aa15-55311477967e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:19.652073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79863f87-834f-4c59-aa15-55311477967e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:19.652689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79863f87-834f-4c59-aa15-55311477967e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:19.700797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c79cd-7d8f-4285-b1f2-7ca475d2c93e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:19.768763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67da95c-ed3d-4dd7-a71b-7f910d62b515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:19.769895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da95c-ed3d-4dd7-a71b-7f910d62b515 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:19.770887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da95c-ed3d-4dd7-a71b-7f910d62b515 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:19.771316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da95c-ed3d-4dd7-a71b-7f910d62b515 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:19.772329 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67da95c-ed3d-4dd7-a71b-7f910d62b515 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:19.772802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67da95c-ed3d-4dd7-a71b-7f910d62b515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:19.774511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67da95c-ed3d-4dd7-a71b-7f910d62b515 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:19.792917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 172/358] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:19.800829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67da95c-ed3d-4dd7-a71b-7f910d62b515 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:19.825681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79863f87-834f-4c59-aa15-55311477967e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:19.829017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79863f87-834f-4c59-aa15-55311477967e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:19.836070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:20.033893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:20.073325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 187/359] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:20.076966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67da95c-ed3d-4dd7-a71b-7f910d62b515 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:20.080304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79863f87-834f-4c59-aa15-55311477967e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:20.164756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529378c8-dd30-425e-b311-75f017da8cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:20.165438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529378c8-dd30-425e-b311-75f017da8cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:20.165724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529378c8-dd30-425e-b311-75f017da8cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:20.166010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529378c8-dd30-425e-b311-75f017da8cd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:20.281139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529378c8-dd30-425e-b311-75f017da8cd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:20.282951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529378c8-dd30-425e-b311-75f017da8cd2 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:20.378423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:20.389057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79863f87-834f-4c59-aa15-55311477967e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:20.490680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f6780-a64d-4255-9914-1f129f5e233f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:20.492083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f6780-a64d-4255-9914-1f129f5e233f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:20.492588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f6780-a64d-4255-9914-1f129f5e233f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:20.493112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f6780-a64d-4255-9914-1f129f5e233f None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:20.494807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f6780-a64d-4255-9914-1f129f5e233f None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:20.496681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f6780-a64d-4255-9914-1f129f5e233f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:20.501191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 188/360] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:20 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:20.501191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f088ca-3fb1-4230-88ca-98c65878ffd8 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:20.501453 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f088ca-3fb1-4230-88ca-98c65878ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:20.502327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f088ca-3fb1-4230-88ca-98c65878ffd8 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:20.505533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 189/361] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:20.509051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f088ca-3fb1-4230-88ca-98c65878ffd8 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:20.548350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:20.636343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 173/362] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 836 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 09:15:20.638971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3f4cb-0f2a-4567-9c2d-087fda78449b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:20.639755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3f4cb-0f2a-4567-9c2d-087fda78449b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:20.640432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3f4cb-0f2a-4567-9c2d-087fda78449b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:20.659481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f088ca-3fb1-4230-88ca-98c65878ffd8 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:20.697634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-07b3f4cb-0f2a-4567-9c2d-087fda78449b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:20.699634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-07b3f4cb-0f2a-4567-9c2d-087fda78449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:20.705419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd691b26-1580-4595-8cad-bbf6e1f094c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:20.706612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd691b26-1580-4595-8cad-bbf6e1f094c1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:20.707030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd691b26-1580-4595-8cad-bbf6e1f094c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:20.707567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd691b26-1580-4595-8cad-bbf6e1f094c1 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:20.709144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd691b26-1580-4595-8cad-bbf6e1f094c1 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:20.710986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd691b26-1580-4595-8cad-bbf6e1f094c1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:20.714778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 190/363] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:20 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:20.716012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd691b26-1580-4595-8cad-bbf6e1f094c1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:20.772225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:20.847072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd691b26-1580-4595-8cad-bbf6e1f094c1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:20.889832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2c95-8240-4097-9ee0-cfec4e37f080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:20.891022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2c95-8240-4097-9ee0-cfec4e37f080 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:20.891498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2c95-8240-4097-9ee0-cfec4e37f080 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:20.891989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d2c95-8240-4097-9ee0-cfec4e37f080 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:20.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d2c95-8240-4097-9ee0-cfec4e37f080 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:20.895735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d2c95-8240-4097-9ee0-cfec4e37f080 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:20.897788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 191/364] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:20 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:20.898874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2c95-8240-4097-9ee0-cfec4e37f080 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:20.927227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 174/365] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:20.929181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3f4cb-0f2a-4567-9c2d-087fda78449b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:21.028079 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d2c95-8240-4097-9ee0-cfec4e37f080 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:21.036011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:21.117378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ed440-4b9b-46c5-acb6-b9cd7ebeffa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:21.118077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed440-4b9b-46c5-acb6-b9cd7ebeffa0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:21.118355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed440-4b9b-46c5-acb6-b9cd7ebeffa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:21.118635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed440-4b9b-46c5-acb6-b9cd7ebeffa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:21.119573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ed440-4b9b-46c5-acb6-b9cd7ebeffa0 None admin] RBAC: Adding query filter params (group.id=129eb944ca1047a1ac4820bd64d84ae8, role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=04cc0f746b104684b7b6e33d52a38126) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:21.119895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ed440-4b9b-46c5-acb6-b9cd7ebeffa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:21.121451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ed440-4b9b-46c5-acb6-b9cd7ebeffa0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:21.136847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 192/366] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:20 2022] GET /identity/v3/role_assignments?group.id=129eb944ca1047a1ac4820bd64d84ae8&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:21.139743 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ed440-4b9b-46c5-acb6-b9cd7ebeffa0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:21.257800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3f4cb-0f2a-4567-9c2d-087fda78449b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:21.308413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:21.309233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ed440-4b9b-46c5-acb6-b9cd7ebeffa0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:21.377484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d060adc-fbda-48e3-881a-a691259163d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:21.378665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d060adc-fbda-48e3-881a-a691259163d4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:21.379166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d060adc-fbda-48e3-881a-a691259163d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:21.379665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d060adc-fbda-48e3-881a-a691259163d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:21.394444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d060adc-fbda-48e3-881a-a691259163d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:21.395635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444f5da-fd2b-48b9-a400-92381a4674be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:21.397419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444f5da-fd2b-48b9-a400-92381a4674be None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:21.397706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444f5da-fd2b-48b9-a400-92381a4674be None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:21.397980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444f5da-fd2b-48b9-a400-92381a4674be None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:21.399392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444f5da-fd2b-48b9-a400-92381a4674be None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:21.401458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444f5da-fd2b-48b9-a400-92381a4674be None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:21.403334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 175/367] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:20 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-4d060adc-fbda-48e3-881a-a691259163d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:21.407959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:21.409052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:15:21.409874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:21.412803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 193/368] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:15:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:21.413102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f17fc-db7c-43ab-a7fe-208fa9f94b91 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:21.419339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f17fc-db7c-43ab-a7fe-208fa9f94b91 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:21.419339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f17fc-db7c-43ab-a7fe-208fa9f94b91 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:21.419339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 176/369] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:21.419782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d060adc-fbda-48e3-881a-a691259163d4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:21.425229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d9005-6ba3-453d-b696-4de547676c94 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:21.425517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d9005-6ba3-453d-b696-4de547676c94 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:21.425801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d9005-6ba3-453d-b696-4de547676c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:21.461085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-c37d9005-6ba3-453d-b696-4de547676c94 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:21.462331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c37d9005-6ba3-453d-b696-4de547676c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:21.512053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:21.618899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 177/370] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:21.621287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d9005-6ba3-453d-b696-4de547676c94 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:21.698797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d060adc-fbda-48e3-881a-a691259163d4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:21.768080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fda0f-11aa-48c1-b832-63ebd43b9580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:21.769442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fda0f-11aa-48c1-b832-63ebd43b9580 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:21.770661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fda0f-11aa-48c1-b832-63ebd43b9580 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:21.771187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fda0f-11aa-48c1-b832-63ebd43b9580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:21.772745 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fda0f-11aa-48c1-b832-63ebd43b9580 None admin] RBAC: Adding query filter params (group.id=129eb944ca1047a1ac4820bd64d84ae8, role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=04cc0f746b104684b7b6e33d52a38126) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:21.773262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fda0f-11aa-48c1-b832-63ebd43b9580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:21.775169 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fda0f-11aa-48c1-b832-63ebd43b9580 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:21.787687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 194/371] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:21 2022] GET /identity/v3/role_assignments?group.id=129eb944ca1047a1ac4820bd64d84ae8&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 260 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:21.789596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72addb2a-16b6-464e-888e-3b6ce6cc4971 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:21.790091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72addb2a-16b6-464e-888e-3b6ce6cc4971 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:21.790536 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72addb2a-16b6-464e-888e-3b6ce6cc4971 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:21.792269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 195/372] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:21.793421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72addb2a-16b6-464e-888e-3b6ce6cc4971 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:21.872570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d9005-6ba3-453d-b696-4de547676c94 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:21.913689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:21.974908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011af6bc-4474-45c6-83ad-25cfc3ecd12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:21.976563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011af6bc-4474-45c6-83ad-25cfc3ecd12a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:21.977327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011af6bc-4474-45c6-83ad-25cfc3ecd12a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:21.978009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011af6bc-4474-45c6-83ad-25cfc3ecd12a None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:21.981965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011af6bc-4474-45c6-83ad-25cfc3ecd12a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:21.983264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011af6bc-4474-45c6-83ad-25cfc3ecd12a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:21.986556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011af6bc-4474-45c6-83ad-25cfc3ecd12a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:22.008591 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 178/373] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:22.012256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01025755-44f3-4ad9-946d-962d9b5f4c3f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:22.013070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01025755-44f3-4ad9-946d-962d9b5f4c3f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:22.013984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01025755-44f3-4ad9-946d-962d9b5f4c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:22.047758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:22.058532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-01025755-44f3-4ad9-946d-962d9b5f4c3f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:22.064339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-01025755-44f3-4ad9-946d-962d9b5f4c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:22.075317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72addb2a-16b6-464e-888e-3b6ce6cc4971 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:22.165548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6184aae-7f36-4502-b5f8-1f2cceb85b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:22.170558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 179/374] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:22.170558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01025755-44f3-4ad9-946d-962d9b5f4c3f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:22.172672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6184aae-7f36-4502-b5f8-1f2cceb85b3e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:22.173755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6184aae-7f36-4502-b5f8-1f2cceb85b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:22.174606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6184aae-7f36-4502-b5f8-1f2cceb85b3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:22.208157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6184aae-7f36-4502-b5f8-1f2cceb85b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-b6184aae-7f36-4502-b5f8-1f2cceb85b3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:22.225145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:22.226836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 196/375] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:22.228965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6184aae-7f36-4502-b5f8-1f2cceb85b3e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:22.496823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01025755-44f3-4ad9-946d-962d9b5f4c3f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:22.516304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:22.550661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc436c7-3e05-4401-b2fc-a7867b892811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:22.551940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc436c7-3e05-4401-b2fc-a7867b892811 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:22.552469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc436c7-3e05-4401-b2fc-a7867b892811 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:22.552755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc436c7-3e05-4401-b2fc-a7867b892811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:22.574641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6184aae-7f36-4502-b5f8-1f2cceb85b3e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:22.630195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5092103-c9b1-4ad0-9951-2943b83fe53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:22.631110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5092103-c9b1-4ad0-9951-2943b83fe53b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:22.631398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5092103-c9b1-4ad0-9951-2943b83fe53b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:22.631718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5092103-c9b1-4ad0-9951-2943b83fe53b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:22.647008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5092103-c9b1-4ad0-9951-2943b83fe53b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-d5092103-c9b1-4ad0-9951-2943b83fe53b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:22.657389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:22.659154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 197/376] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:22.660822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5092103-c9b1-4ad0-9951-2943b83fe53b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:22.681785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:22.690560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc436c7-3e05-4401-b2fc-a7867b892811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:22.694423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc436c7-3e05-4401-b2fc-a7867b892811 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:22.825114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5092103-c9b1-4ad0-9951-2943b83fe53b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:22.895925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7001f-216b-43b5-a493-501c35d42104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:22.897714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7001f-216b-43b5-a493-501c35d42104 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:22.898460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7001f-216b-43b5-a493-501c35d42104 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:22.899205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7001f-216b-43b5-a493-501c35d42104 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:22.901343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7001f-216b-43b5-a493-501c35d42104 None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:22.902244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7001f-216b-43b5-a493-501c35d42104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:22.905270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7001f-216b-43b5-a493-501c35d42104 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:22.926397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 198/377] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:22.929447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca9f23-72f0-41c8-a003-ef6a7d5ee66f None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:22.930329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca9f23-72f0-41c8-a003-ef6a7d5ee66f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:22.931100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca9f23-72f0-41c8-a003-ef6a7d5ee66f None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:22.931706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 180/378] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:22.933167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc436c7-3e05-4401-b2fc-a7867b892811 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:22.935572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 199/379] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:22.937400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca9f23-72f0-41c8-a003-ef6a7d5ee66f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:22.951078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:23.142123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:23.180282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc436c7-3e05-4401-b2fc-a7867b892811 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:23.205126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca9f23-72f0-41c8-a003-ef6a7d5ee66f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:23.307736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a30bf2-4321-4c99-b25e-885546182a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:23.309398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a30bf2-4321-4c99-b25e-885546182a07 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:23.310192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a30bf2-4321-4c99-b25e-885546182a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:23.310472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7990d3-998c-431a-8d10-fde815f53cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:23.311228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a30bf2-4321-4c99-b25e-885546182a07 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:23.312165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7990d3-998c-431a-8d10-fde815f53cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:23.312718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7990d3-998c-431a-8d10-fde815f53cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:23.313227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7990d3-998c-431a-8d10-fde815f53cf5 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:23.313983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a30bf2-4321-4c99-b25e-885546182a07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:23.314827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a30bf2-4321-4c99-b25e-885546182a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:23.315744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7990d3-998c-431a-8d10-fde815f53cf5 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:23.318658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a30bf2-4321-4c99-b25e-885546182a07 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:23.319283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7990d3-998c-431a-8d10-fde815f53cf5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:23.323389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 200/380] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:22 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:23.328341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7990d3-998c-431a-8d10-fde815f53cf5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:23.346021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 181/381] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:23.347450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821006f8-d5b7-4b53-8fd4-40405266114c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:23.347877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821006f8-d5b7-4b53-8fd4-40405266114c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:23.348219 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821006f8-d5b7-4b53-8fd4-40405266114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:23.368974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-821006f8-d5b7-4b53-8fd4-40405266114c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:23.370272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-821006f8-d5b7-4b53-8fd4-40405266114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:23.459881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 182/382] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:23.460659 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821006f8-d5b7-4b53-8fd4-40405266114c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:23.472263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:23.544826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7990d3-998c-431a-8d10-fde815f53cf5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:23.615352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821006f8-d5b7-4b53-8fd4-40405266114c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:23.621161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69314126-7d2d-4932-8226-c0c07ef2e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:23.622709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69314126-7d2d-4932-8226-c0c07ef2e748 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:23.623215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69314126-7d2d-4932-8226-c0c07ef2e748 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:23.623715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69314126-7d2d-4932-8226-c0c07ef2e748 None admin] PATH_INFO: `/v3/projects/d706bacf3a904a118b091c4d80341fee` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:23.626324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69314126-7d2d-4932-8226-c0c07ef2e748 None admin] RBAC: Authorizing `identity:get_project(project_id=d706bacf3a904a118b091c4d80341fee)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:23.629765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69314126-7d2d-4932-8226-c0c07ef2e748 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:23.633371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 201/383] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:23 2022] GET /identity/v3/projects/d706bacf3a904a118b091c4d80341fee => generated 344 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:23.634840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69314126-7d2d-4932-8226-c0c07ef2e748 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:23.660276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ae8fc-19f5-4cc4-a461-f44df6ab0b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:23.661256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ae8fc-19f5-4cc4-a461-f44df6ab0b17 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:23.661538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ae8fc-19f5-4cc4-a461-f44df6ab0b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:23.661814 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ae8fc-19f5-4cc4-a461-f44df6ab0b17 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:23.663285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ae8fc-19f5-4cc4-a461-f44df6ab0b17 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:23.665007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ae8fc-19f5-4cc4-a461-f44df6ab0b17 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:23.668021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:23.669099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 183/384] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:23 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:23.669764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ae8fc-19f5-4cc4-a461-f44df6ab0b17 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:23.813033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69314126-7d2d-4932-8226-c0c07ef2e748 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:23.817229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:23.890753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ae8fc-19f5-4cc4-a461-f44df6ab0b17 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:23.893860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcfa54-40aa-4f08-986d-aea431f25aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:23.896427 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcfa54-40aa-4f08-986d-aea431f25aef None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:23.897154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcfa54-40aa-4f08-986d-aea431f25aef None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:23.897885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcfa54-40aa-4f08-986d-aea431f25aef None admin] PATH_INFO: `/v3/users/6a53d8205fb64099901af9469b01f1a5` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:23.900881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcfa54-40aa-4f08-986d-aea431f25aef None admin] RBAC: Authorizing `identity:get_user(user_id=6a53d8205fb64099901af9469b01f1a5)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:23.903878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcfa54-40aa-4f08-986d-aea431f25aef None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:23.907389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 202/385] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:15:23 2022] GET /identity/v3/users/6a53d8205fb64099901af9469b01f1a5 => generated 293 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:23.913398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcfa54-40aa-4f08-986d-aea431f25aef None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:24.006615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664872ef-1901-4aa9-a41c-fd5a25f18af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:24.008340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664872ef-1901-4aa9-a41c-fd5a25f18af0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:24.008957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664872ef-1901-4aa9-a41c-fd5a25f18af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:24.009564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664872ef-1901-4aa9-a41c-fd5a25f18af0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:24.029981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664872ef-1901-4aa9-a41c-fd5a25f18af0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-664872ef-1901-4aa9-a41c-fd5a25f18af0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:24.042074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:24.043817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:24.045529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 184/386] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:15:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:24.046517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664872ef-1901-4aa9-a41c-fd5a25f18af0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:24.101190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:24.131815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcfa54-40aa-4f08-986d-aea431f25aef None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:24.198392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed961a-0c8a-4654-b5e1-aaa3289f5940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:24.198392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed961a-0c8a-4654-b5e1-aaa3289f5940 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:24.198392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed961a-0c8a-4654-b5e1-aaa3289f5940 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:24.198392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed961a-0c8a-4654-b5e1-aaa3289f5940 None admin] PATH_INFO: `/v3/groups/43a7ae9398ab4e78a9ab45fc05d930e2` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:24.205444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed961a-0c8a-4654-b5e1-aaa3289f5940 None admin] RBAC: Authorizing `identity:get_group(group_id=43a7ae9398ab4e78a9ab45fc05d930e2)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:24.206972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed961a-0c8a-4654-b5e1-aaa3289f5940 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:24.210763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 203/387] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:23 2022] GET /identity/v3/groups/43a7ae9398ab4e78a9ab45fc05d930e2 => generated 229 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:24.211910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed961a-0c8a-4654-b5e1-aaa3289f5940 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:24.280061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664872ef-1901-4aa9-a41c-fd5a25f18af0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:24.339802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed961a-0c8a-4654-b5e1-aaa3289f5940 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:24.340803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:24.364361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e136f-abcb-4ccf-916a-b5c0277c1189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:24.366382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e136f-abcb-4ccf-916a-b5c0277c1189 None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:24.367107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e136f-abcb-4ccf-916a-b5c0277c1189 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:24.368187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e136f-abcb-4ccf-916a-b5c0277c1189 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97/groups/129eb944ca1047a1ac4820bd64d84ae8/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:24.372688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e136f-abcb-4ccf-916a-b5c0277c1189 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c495a22d3f7484cb377da68dab0bf97, group_id=129eb944ca1047a1ac4820bd64d84ae8, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:24.376272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e136f-abcb-4ccf-916a-b5c0277c1189 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:24.381768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411edc0-2dc9-4210-8354-d8a2a403e021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:24.382508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411edc0-2dc9-4210-8354-d8a2a403e021 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:24.382796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411edc0-2dc9-4210-8354-d8a2a403e021 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:24.383126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411edc0-2dc9-4210-8354-d8a2a403e021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:24.384641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411edc0-2dc9-4210-8354-d8a2a403e021 None admin] RBAC: Adding query filter params (role.id=536229a47a9d4a03b6d0b1c59fd8f39a, user.id=6a53d8205fb64099901af9469b01f1a5, scope.system=all) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:24.384909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411edc0-2dc9-4210-8354-d8a2a403e021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:24.386407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411edc0-2dc9-4210-8354-d8a2a403e021 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:24.398867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:24.407879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 204/388] 10.222.0.64 () {56 vars in 1308 bytes} [Mon Jun 27 09:15:24 2022] GET /identity/v3/role_assignments?user.id=6a53d8205fb64099901af9469b01f1a5&scope.system=all&role.id=536229a47a9d4a03b6d0b1c59fd8f39a => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:24.407879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411edc0-2dc9-4210-8354-d8a2a403e021 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:24.460744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 185/389] 10.222.0.64 () {58 vars in 1442 bytes} [Mon Jun 27 09:15:24 2022] PUT /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97/groups/129eb944ca1047a1ac4820bd64d84ae8/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:24.466811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e136f-abcb-4ccf-916a-b5c0277c1189 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:24.547362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411edc0-2dc9-4210-8354-d8a2a403e021 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:24.592639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2eddf0-80a6-45e8-b5c4-d37c7c6ff3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:24.593444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eddf0-80a6-45e8-b5c4-d37c7c6ff3bd None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:24.593763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eddf0-80a6-45e8-b5c4-d37c7c6ff3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:24.594159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2eddf0-80a6-45e8-b5c4-d37c7c6ff3bd None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:24.595459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2eddf0-80a6-45e8-b5c4-d37c7c6ff3bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:24.595459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2eddf0-80a6-45e8-b5c4-d37c7c6ff3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:24.596887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2eddf0-80a6-45e8-b5c4-d37c7c6ff3bd None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:24.608768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 205/390] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:24.610007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210a3c4-cf48-4ffe-ba32-51e98ee31171 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:24.615744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210a3c4-cf48-4ffe-ba32-51e98ee31171 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:24.615744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210a3c4-cf48-4ffe-ba32-51e98ee31171 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:24.624652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 206/391] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:24 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:24.625576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210a3c4-cf48-4ffe-ba32-51e98ee31171 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:24.748222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:24.767925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e136f-abcb-4ccf-916a-b5c0277c1189 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:24.804195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210a3c4-cf48-4ffe-ba32-51e98ee31171 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:24.839968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e944a5-7537-47bf-8808-a163f2aa0ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:24.841092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e944a5-7537-47bf-8808-a163f2aa0ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:24.841561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e944a5-7537-47bf-8808-a163f2aa0ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:24.842030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e944a5-7537-47bf-8808-a163f2aa0ad3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:24.843625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e944a5-7537-47bf-8808-a163f2aa0ad3 None admin] RBAC: Adding query filter params (group.id=43a7ae9398ab4e78a9ab45fc05d930e2, role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=d706bacf3a904a118b091c4d80341fee) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:24.844169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e944a5-7537-47bf-8808-a163f2aa0ad3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:24.846955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e944a5-7537-47bf-8808-a163f2aa0ad3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:24.850183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a707f1e-531a-49f0-89c1-c34cf73af697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:24.851077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707f1e-531a-49f0-89c1-c34cf73af697 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:24.851363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707f1e-531a-49f0-89c1-c34cf73af697 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:24.851683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707f1e-531a-49f0-89c1-c34cf73af697 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:24.853149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a707f1e-531a-49f0-89c1-c34cf73af697 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:24.854762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a707f1e-531a-49f0-89c1-c34cf73af697 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:24.857275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 207/392] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:24 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:24.857493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 186/393] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:24 2022] GET /identity/v3/role_assignments?group.id=43a7ae9398ab4e78a9ab45fc05d930e2&scope.project.id=d706bacf3a904a118b091c4d80341fee&role.id=03b724e03f704f2ab338b64025a3db6d => generated 260 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:24.858187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9b782-d492-4de0-93bd-0e13788440ae None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:24.858522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9b782-d492-4de0-93bd-0e13788440ae None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:24.858821 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9b782-d492-4de0-93bd-0e13788440ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:24.871615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e944a5-7537-47bf-8808-a163f2aa0ad3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:24.880824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-32f9b782-d492-4de0-93bd-0e13788440ae None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:24.882246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-32f9b782-d492-4de0-93bd-0e13788440ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:25.029858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:25.062641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 208/394] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:25.149554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e944a5-7537-47bf-8808-a163f2aa0ad3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:25.151710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9b782-d492-4de0-93bd-0e13788440ae None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:25.267283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08c5c8-9de3-46cf-bcdc-92d43ef471b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:25.270870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c5c8-9de3-46cf-bcdc-92d43ef471b8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:25.270870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c5c8-9de3-46cf-bcdc-92d43ef471b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:25.270870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08c5c8-9de3-46cf-bcdc-92d43ef471b8 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:25.281917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08c5c8-9de3-46cf-bcdc-92d43ef471b8 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:25.286834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08c5c8-9de3-46cf-bcdc-92d43ef471b8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:25.288441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 187/395] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:24 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:25.305714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08c5c8-9de3-46cf-bcdc-92d43ef471b8 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:25.383043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:25.514938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9b782-d492-4de0-93bd-0e13788440ae None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:25.608217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb287c-ff1b-4df6-859b-8f17a32f9dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:25.609559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb287c-ff1b-4df6-859b-8f17a32f9dca None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:25.610184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb287c-ff1b-4df6-859b-8f17a32f9dca None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:25.610669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb287c-ff1b-4df6-859b-8f17a32f9dca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:25.617935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08c5c8-9de3-46cf-bcdc-92d43ef471b8 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:25.635563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb287c-ff1b-4df6-859b-8f17a32f9dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-7afb287c-ff1b-4df6-859b-8f17a32f9dca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:25.653254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:25.655003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:25.656654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:25.657826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 209/396] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 506 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:15:25.664186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:25.669121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb287c-ff1b-4df6-859b-8f17a32f9dca None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:25.712147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23124f0-2e57-4747-b509-ab446ebe914b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:25.713095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23124f0-2e57-4747-b509-ab446ebe914b None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:25.713393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23124f0-2e57-4747-b509-ab446ebe914b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:25.713680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23124f0-2e57-4747-b509-ab446ebe914b None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97/groups/129eb944ca1047a1ac4820bd64d84ae8/roles/296546e6917a4eb6b58cac525a76f8a6` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:25.716552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23124f0-2e57-4747-b509-ab446ebe914b None admin] RBAC: Authorizing `identity:create_grant(project_id=8c495a22d3f7484cb377da68dab0bf97, group_id=129eb944ca1047a1ac4820bd64d84ae8, role_id=296546e6917a4eb6b58cac525a76f8a6)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:25.718105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23124f0-2e57-4747-b509-ab446ebe914b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:25.813837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 188/397] 10.222.0.64 () {58 vars in 1442 bytes} [Mon Jun 27 09:15:25 2022] PUT /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97/groups/129eb944ca1047a1ac4820bd64d84ae8/roles/296546e6917a4eb6b58cac525a76f8a6 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:25.815185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7a353-b3c8-4e22-8b11-87a18d86a55a None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:25.815649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7a353-b3c8-4e22-8b11-87a18d86a55a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:25.815948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7a353-b3c8-4e22-8b11-87a18d86a55a None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:25.824168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 189/398] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:25.830007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c2edc-a817-43ec-91a6-bcc91309cd1d None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:25.830319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c2edc-a817-43ec-91a6-bcc91309cd1d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:25.830662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c2edc-a817-43ec-91a6-bcc91309cd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:25.866837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b16c2edc-a817-43ec-91a6-bcc91309cd1d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:25.868725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b16c2edc-a817-43ec-91a6-bcc91309cd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:25.875762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb287c-ff1b-4df6-859b-8f17a32f9dca None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:25.919674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:25.946292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64c683-2e3a-4be2-9cdf-ec069a69915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:25.947458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64c683-2e3a-4be2-9cdf-ec069a69915e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:25.948013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64c683-2e3a-4be2-9cdf-ec069a69915e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:25.948502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64c683-2e3a-4be2-9cdf-ec069a69915e None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:25.950057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64c683-2e3a-4be2-9cdf-ec069a69915e None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:25.950765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64c683-2e3a-4be2-9cdf-ec069a69915e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:25.953550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64c683-2e3a-4be2-9cdf-ec069a69915e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:25.979824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 210/399] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:25.984081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64c683-2e3a-4be2-9cdf-ec069a69915e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:26.079155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 190/400] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:26.113582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c2edc-a817-43ec-91a6-bcc91309cd1d None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:26.188673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64c683-2e3a-4be2-9cdf-ec069a69915e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:26.295388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07183-511e-4c21-b39b-57347a3566c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:26.300890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07183-511e-4c21-b39b-57347a3566c3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:26.302295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07183-511e-4c21-b39b-57347a3566c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:26.304160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07183-511e-4c21-b39b-57347a3566c3 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:26.308471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b07183-511e-4c21-b39b-57347a3566c3 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:26.311968 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b07183-511e-4c21-b39b-57347a3566c3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:26.321283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 211/401] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:25 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:26.327573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07183-511e-4c21-b39b-57347a3566c3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:26.396056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c2edc-a817-43ec-91a6-bcc91309cd1d None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:26.426015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:26.426842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:26.505565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b5984-45ed-455d-807c-13112f352a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:26.506360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b5984-45ed-455d-807c-13112f352a88 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:26.506641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b5984-45ed-455d-807c-13112f352a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:26.506920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b5984-45ed-455d-807c-13112f352a88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:26.531572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b5984-45ed-455d-807c-13112f352a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:26.545311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07183-511e-4c21-b39b-57347a3566c3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-ef1b5984-45ed-455d-807c-13112f352a88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:26.549253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:26.551467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:26.553032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:26.557953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 191/402] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:15:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:15:26.563592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b5984-45ed-455d-807c-13112f352a88 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:26.639973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f44b3e-5259-4495-b2b9-7ee5f1eb2e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:26.642223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f44b3e-5259-4495-b2b9-7ee5f1eb2e39 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:26.643138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f44b3e-5259-4495-b2b9-7ee5f1eb2e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:26.643900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f44b3e-5259-4495-b2b9-7ee5f1eb2e39 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:26.649282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f44b3e-5259-4495-b2b9-7ee5f1eb2e39 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:26.652929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f44b3e-5259-4495-b2b9-7ee5f1eb2e39 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:26.664229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 212/403] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:26 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:26.673705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f44b3e-5259-4495-b2b9-7ee5f1eb2e39 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:26.726677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b5984-45ed-455d-807c-13112f352a88 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:26.800162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeddc5-c89b-4de9-978e-8d032675b8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:26.801389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeddc5-c89b-4de9-978e-8d032675b8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:26.801801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeddc5-c89b-4de9-978e-8d032675b8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:26.802241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeddc5-c89b-4de9-978e-8d032675b8c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:26.803056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeddc5-c89b-4de9-978e-8d032675b8c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:26.803368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeddc5-c89b-4de9-978e-8d032675b8c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:26.804907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeddc5-c89b-4de9-978e-8d032675b8c3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:26.825097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 192/404] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:26.832700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeddc5-c89b-4de9-978e-8d032675b8c3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:26.875607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:26.923521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f44b3e-5259-4495-b2b9-7ee5f1eb2e39 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:26.964951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:27.007479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e27f2a-c10d-4154-9b7d-5235aa5cb4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:27.011112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e27f2a-c10d-4154-9b7d-5235aa5cb4d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:27.012333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e27f2a-c10d-4154-9b7d-5235aa5cb4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:27.018667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e27f2a-c10d-4154-9b7d-5235aa5cb4d6 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126/groups/129eb944ca1047a1ac4820bd64d84ae8/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:27.023434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e27f2a-c10d-4154-9b7d-5235aa5cb4d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=04cc0f746b104684b7b6e33d52a38126, group_id=129eb944ca1047a1ac4820bd64d84ae8, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:27.033280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e27f2a-c10d-4154-9b7d-5235aa5cb4d6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:27.060992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeddc5-c89b-4de9-978e-8d032675b8c3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:27.129524 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3b912-7e94-40ba-b64f-4aa1e3ecb572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:27.131623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3b912-7e94-40ba-b64f-4aa1e3ecb572 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:27.132427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3b912-7e94-40ba-b64f-4aa1e3ecb572 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:27.133195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3b912-7e94-40ba-b64f-4aa1e3ecb572 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:27.135924 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3b912-7e94-40ba-b64f-4aa1e3ecb572 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:27.138997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3b912-7e94-40ba-b64f-4aa1e3ecb572 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:27.143043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 193/405] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:26 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:27.149996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3b912-7e94-40ba-b64f-4aa1e3ecb572 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:27.172814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 213/406] 10.222.0.64 () {58 vars in 1442 bytes} [Mon Jun 27 09:15:26 2022] PUT /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126/groups/129eb944ca1047a1ac4820bd64d84ae8/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:15:27.370809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:27.383138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3b912-7e94-40ba-b64f-4aa1e3ecb572 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:27.458467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163aae9-c98d-4673-acf1-d609ad3423a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:27.460339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163aae9-c98d-4673-acf1-d609ad3423a1 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:27.461075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163aae9-c98d-4673-acf1-d609ad3423a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:27.461888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163aae9-c98d-4673-acf1-d609ad3423a1 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:27.464494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163aae9-c98d-4673-acf1-d609ad3423a1 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:27.467808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163aae9-c98d-4673-acf1-d609ad3423a1 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:27.471287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 194/407] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:27 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:27.478237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e27f2a-c10d-4154-9b7d-5235aa5cb4d6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:27.690000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e27f2a-c10d-4154-9b7d-5235aa5cb4d6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:27.779506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f2643-3905-43dc-9062-6cabf50a8bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:27.780458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f2643-3905-43dc-9062-6cabf50a8bac None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:27.780742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f2643-3905-43dc-9062-6cabf50a8bac None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:27.781014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f2643-3905-43dc-9062-6cabf50a8bac None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126/groups/129eb944ca1047a1ac4820bd64d84ae8/roles/296546e6917a4eb6b58cac525a76f8a6` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:27.783566 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f2643-3905-43dc-9062-6cabf50a8bac None admin] RBAC: Authorizing `identity:create_grant(project_id=04cc0f746b104684b7b6e33d52a38126, group_id=129eb944ca1047a1ac4820bd64d84ae8, role_id=296546e6917a4eb6b58cac525a76f8a6)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:27.785254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f2643-3905-43dc-9062-6cabf50a8bac None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:27.873322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 214/408] 10.222.0.64 () {58 vars in 1442 bytes} [Mon Jun 27 09:15:27 2022] PUT /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126/groups/129eb944ca1047a1ac4820bd64d84ae8/roles/296546e6917a4eb6b58cac525a76f8a6 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:27.970459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645180f-e2c1-4c9f-afbf-cc8073d04025 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:27.971254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645180f-e2c1-4c9f-afbf-cc8073d04025 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:27.971254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645180f-e2c1-4c9f-afbf-cc8073d04025 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:27.974249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 195/409] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:27.980274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c78ef-4942-46f0-87b0-44ea9d831eee None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:27.980680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c78ef-4942-46f0-87b0-44ea9d831eee None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:27.980854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c78ef-4942-46f0-87b0-44ea9d831eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:28.004638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ab4c78ef-4942-46f0-87b0-44ea9d831eee None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:28.006141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ab4c78ef-4942-46f0-87b0-44ea9d831eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:28.055055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:28.108052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 215/410] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:28.148350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645180f-e2c1-4c9f-afbf-cc8073d04025 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:28.231963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e5da1-294b-43f8-a286-506af6298c12 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:28.232428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e5da1-294b-43f8-a286-506af6298c12 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:28.232904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e5da1-294b-43f8-a286-506af6298c12 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:28.236246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 216/411] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:28.245756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864aea8a-d853-4dd5-b3ab-416ee39f0161 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:28.246218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864aea8a-d853-4dd5-b3ab-416ee39f0161 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:28.246700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864aea8a-d853-4dd5-b3ab-416ee39f0161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:28.284694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-864aea8a-d853-4dd5-b3ab-416ee39f0161 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:28.291219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-864aea8a-d853-4dd5-b3ab-416ee39f0161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:28.402165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645180f-e2c1-4c9f-afbf-cc8073d04025 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:28.475796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 217/412] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:15:28.518556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bd4f3-93c6-4eca-b474-06217e94cba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:28.520368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bd4f3-93c6-4eca-b474-06217e94cba7 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:28.521087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bd4f3-93c6-4eca-b474-06217e94cba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:28.521778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bd4f3-93c6-4eca-b474-06217e94cba7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:28.527102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864aea8a-d853-4dd5-b3ab-416ee39f0161 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:28.544145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bd4f3-93c6-4eca-b474-06217e94cba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-185bd4f3-93c6-4eca-b474-06217e94cba7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:28.561416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:28.563104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:28.564641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:28.569786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 196/413] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:28.594634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bd4f3-93c6-4eca-b474-06217e94cba7 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:28.645147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:28.823381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bd4f3-93c6-4eca-b474-06217e94cba7 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:28.898038 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864aea8a-d853-4dd5-b3ab-416ee39f0161 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:28.905073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9e031-7cff-4c8d-840a-e169ffe02810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:28.905803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:28.907965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9e031-7cff-4c8d-840a-e169ffe02810 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:28.910927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9e031-7cff-4c8d-840a-e169ffe02810 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:28.911817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9e031-7cff-4c8d-840a-e169ffe02810 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:28.917612 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9e031-7cff-4c8d-840a-e169ffe02810 None admin] RBAC: Adding query filter params (name=member) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:28.920902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9e031-7cff-4c8d-840a-e169ffe02810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:28.921613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9e031-7cff-4c8d-840a-e169ffe02810 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:28.939234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 197/414] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:28.949601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9e031-7cff-4c8d-840a-e169ffe02810 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:29.018033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8563f87-d7a9-4788-be93-08a91cd8ba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:29.022554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563f87-d7a9-4788-be93-08a91cd8ba3d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:29.025400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563f87-d7a9-4788-be93-08a91cd8ba3d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:29.031347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8563f87-d7a9-4788-be93-08a91cd8ba3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:29.070646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8563f87-d7a9-4788-be93-08a91cd8ba3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-a8563f87-d7a9-4788-be93-08a91cd8ba3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:29.102464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:29.104201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:15:29.106442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:29.118399 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 218/415] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 592 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:15:29.130472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8563f87-d7a9-4788-be93-08a91cd8ba3d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:29.277984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:29.314663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9e031-7cff-4c8d-840a-e169ffe02810 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:29.381121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ec00b-7693-41ee-8e84-0a07522c93a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:29.382579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ec00b-7693-41ee-8e84-0a07522c93a4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:29.383314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ec00b-7693-41ee-8e84-0a07522c93a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:29.384059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ec00b-7693-41ee-8e84-0a07522c93a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:29.433854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8563f87-d7a9-4788-be93-08a91cd8ba3d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:29.497837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f823ea-6ee0-411c-96eb-2bcc8343074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:29.498537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f823ea-6ee0-411c-96eb-2bcc8343074c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:29.499005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f823ea-6ee0-411c-96eb-2bcc8343074c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:29.499472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f823ea-6ee0-411c-96eb-2bcc8343074c None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:29.500966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f823ea-6ee0-411c-96eb-2bcc8343074c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:29.501473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f823ea-6ee0-411c-96eb-2bcc8343074c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:29.504551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f823ea-6ee0-411c-96eb-2bcc8343074c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:29.527072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 219/416] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:29.527072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9f22c-e04f-4b22-8392-416fd887dedd None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:29.527072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9f22c-e04f-4b22-8392-416fd887dedd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:29.527072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9f22c-e04f-4b22-8392-416fd887dedd None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:29.528371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 220/417] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:29.534816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ec00b-7693-41ee-8e84-0a07522c93a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:29.536263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677687b5-06fb-4925-bdd4-c7e246899041 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:29.536729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677687b5-06fb-4925-bdd4-c7e246899041 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:29.537184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677687b5-06fb-4925-bdd4-c7e246899041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:29.540429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ec00b-7693-41ee-8e84-0a07522c93a4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:29.567457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-677687b5-06fb-4925-bdd4-c7e246899041 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:29.569791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-677687b5-06fb-4925-bdd4-c7e246899041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:29.598279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:29.708236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:29.709089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 221/418] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:15:29.710545 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677687b5-06fb-4925-bdd4-c7e246899041 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:29.739835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 198/419] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 792 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 09:15:29.747307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ec00b-7693-41ee-8e84-0a07522c93a4 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:29.878295 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ec00b-7693-41ee-8e84-0a07522c93a4 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:29.916881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677687b5-06fb-4925-bdd4-c7e246899041 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:29.948114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140908c-1a34-45f4-8586-f31a86999800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:29.950138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140908c-1a34-45f4-8586-f31a86999800 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:29.950665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140908c-1a34-45f4-8586-f31a86999800 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:29.951251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140908c-1a34-45f4-8586-f31a86999800 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:29.954309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140908c-1a34-45f4-8586-f31a86999800 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:29.957160 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140908c-1a34-45f4-8586-f31a86999800 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:29.965784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 199/420] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:29 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:29.965784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140908c-1a34-45f4-8586-f31a86999800 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:29.968800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0907e-fad9-4d79-a85f-2029e970f3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:29.970975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0907e-fad9-4d79-a85f-2029e970f3cb None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:29.971486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0907e-fad9-4d79-a85f-2029e970f3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:29.971977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0907e-fad9-4d79-a85f-2029e970f3cb None admin] PATH_INFO: `/v3/groups/43a7ae9398ab4e78a9ab45fc05d930e2` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:29.974063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0907e-fad9-4d79-a85f-2029e970f3cb None admin] RBAC: Authorizing `identity:get_group(group_id=43a7ae9398ab4e78a9ab45fc05d930e2)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:29.975802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0907e-fad9-4d79-a85f-2029e970f3cb None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:29.977784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 222/421] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:29 2022] GET /identity/v3/groups/43a7ae9398ab4e78a9ab45fc05d930e2 => generated 229 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:29.978795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0907e-fad9-4d79-a85f-2029e970f3cb None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:30.008959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:30.081520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:30.120913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0907e-fad9-4d79-a85f-2029e970f3cb None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:30.167271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a85fa-9f79-4472-bb61-275d6974e6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:30.168748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a85fa-9f79-4472-bb61-275d6974e6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:30.169301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a85fa-9f79-4472-bb61-275d6974e6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:30.169724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a85fa-9f79-4472-bb61-275d6974e6b8 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:30.171289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a85fa-9f79-4472-bb61-275d6974e6b8 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:30.173755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a85fa-9f79-4472-bb61-275d6974e6b8 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:30.176753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 223/422] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:29 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:30.181737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140908c-1a34-45f4-8586-f31a86999800 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:30.183318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a85fa-9f79-4472-bb61-275d6974e6b8 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:30.240064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9cf10-1bc5-4bab-9f54-3d15d238799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:30.241460 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9cf10-1bc5-4bab-9f54-3d15d238799e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:30.241912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9cf10-1bc5-4bab-9f54-3d15d238799e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:30.242441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9cf10-1bc5-4bab-9f54-3d15d238799e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:30.265244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9cf10-1bc5-4bab-9f54-3d15d238799e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-80f9cf10-1bc5-4bab-9f54-3d15d238799e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:30.274290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:30.275586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:30.276451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:30.285570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 200/423] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:15:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:30.285570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9cf10-1bc5-4bab-9f54-3d15d238799e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:30.335673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:30.356634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a85fa-9f79-4472-bb61-275d6974e6b8 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:30.402550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a18a3-0d5d-4207-af4e-d47afaf62898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:30.404024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a18a3-0d5d-4207-af4e-d47afaf62898 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:30.404492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a18a3-0d5d-4207-af4e-d47afaf62898 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:30.405415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a18a3-0d5d-4207-af4e-d47afaf62898 None admin] PATH_INFO: `/v3/projects/d706bacf3a904a118b091c4d80341fee` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:30.407321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a18a3-0d5d-4207-af4e-d47afaf62898 None admin] RBAC: Authorizing `identity:get_project(project_id=d706bacf3a904a118b091c4d80341fee)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:30.409071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a18a3-0d5d-4207-af4e-d47afaf62898 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:30.415568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 224/424] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:30 2022] GET /identity/v3/projects/d706bacf3a904a118b091c4d80341fee => generated 344 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:30.415568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a18a3-0d5d-4207-af4e-d47afaf62898 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:30.492737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9cf10-1bc5-4bab-9f54-3d15d238799e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:30.527931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:30.571312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea88ae6-168b-430f-a05b-70c4f8e09d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:30.573163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea88ae6-168b-430f-a05b-70c4f8e09d43 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:30.573982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea88ae6-168b-430f-a05b-70c4f8e09d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:30.574920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea88ae6-168b-430f-a05b-70c4f8e09d43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:30.577082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea88ae6-168b-430f-a05b-70c4f8e09d43 None admin] RBAC: Adding query filter params (group.id=129eb944ca1047a1ac4820bd64d84ae8, role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=8c495a22d3f7484cb377da68dab0bf97) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:30.577966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea88ae6-168b-430f-a05b-70c4f8e09d43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:30.580712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea88ae6-168b-430f-a05b-70c4f8e09d43 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:30.592778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 201/425] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:30 2022] GET /identity/v3/role_assignments?group.id=129eb944ca1047a1ac4820bd64d84ae8&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 611 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:30.594728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea88ae6-168b-430f-a05b-70c4f8e09d43 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:30.619194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a18a3-0d5d-4207-af4e-d47afaf62898 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:30.728699 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abe25e-2e10-4775-91bd-9c5731659fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:30.729863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abe25e-2e10-4775-91bd-9c5731659fba None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:30.730371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abe25e-2e10-4775-91bd-9c5731659fba None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:30.731006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abe25e-2e10-4775-91bd-9c5731659fba None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:30.732371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abe25e-2e10-4775-91bd-9c5731659fba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:30.732872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abe25e-2e10-4775-91bd-9c5731659fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:30.735473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abe25e-2e10-4775-91bd-9c5731659fba None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:30.771262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 225/426] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:30.780238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abe25e-2e10-4775-91bd-9c5731659fba None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:30.814976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea88ae6-168b-430f-a05b-70c4f8e09d43 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:30.836172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:30.877775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5a9a7-f883-4d0e-aef2-9c8418ca3623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:30.880545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5a9a7-f883-4d0e-aef2-9c8418ca3623 None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:30.881128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5a9a7-f883-4d0e-aef2-9c8418ca3623 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:30.881635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5a9a7-f883-4d0e-aef2-9c8418ca3623 None admin] PATH_INFO: `/v3/projects/d706bacf3a904a118b091c4d80341fee/groups/43a7ae9398ab4e78a9ab45fc05d930e2/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:30.885580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5a9a7-f883-4d0e-aef2-9c8418ca3623 None admin] RBAC: Authorizing `identity:create_grant(project_id=d706bacf3a904a118b091c4d80341fee, group_id=43a7ae9398ab4e78a9ab45fc05d930e2, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:30.887458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5a9a7-f883-4d0e-aef2-9c8418ca3623 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:30.981694 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 202/427] 10.222.0.64 () {58 vars in 1442 bytes} [Mon Jun 27 09:15:30 2022] PUT /identity/v3/projects/d706bacf3a904a118b091c4d80341fee/groups/43a7ae9398ab4e78a9ab45fc05d930e2/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:30.985747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cd5c0-e564-4010-825c-35ff317cfacb None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:30.986256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cd5c0-e564-4010-825c-35ff317cfacb None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:30.986745 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cd5c0-e564-4010-825c-35ff317cfacb None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:30.989543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abe25e-2e10-4775-91bd-9c5731659fba None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:30.990003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 203/428] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:30.991701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b961f-cc4e-4806-a7a4-004ce9274af3 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:30.991991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b961f-cc4e-4806-a7a4-004ce9274af3 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:30.992301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b961f-cc4e-4806-a7a4-004ce9274af3 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:30.995050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 204/429] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:30.998493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4c6bf-63b5-4f97-8c50-5b8d6e58d1fc None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:30.998784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4c6bf-63b5-4f97-8c50-5b8d6e58d1fc None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:30.999066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4c6bf-63b5-4f97-8c50-5b8d6e58d1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:31.023194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-c5e4c6bf-63b5-4f97-8c50-5b8d6e58d1fc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:31.025618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c5e4c6bf-63b5-4f97-8c50-5b8d6e58d1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:31.071002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85d82d-357c-48ac-9774-6d2ac61556a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:31.071966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85d82d-357c-48ac-9774-6d2ac61556a1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:31.072277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85d82d-357c-48ac-9774-6d2ac61556a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:31.072542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85d82d-357c-48ac-9774-6d2ac61556a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:31.080870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:31.159250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 205/430] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 09:15:31.160699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53c9e2-3434-44a5-92f0-877024c20264 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:31.161134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53c9e2-3434-44a5-92f0-877024c20264 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:31.161283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53c9e2-3434-44a5-92f0-877024c20264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:31.186559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85d82d-357c-48ac-9774-6d2ac61556a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:31.188356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85d82d-357c-48ac-9774-6d2ac61556a1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:31.189178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ae53c9e2-3434-44a5-92f0-877024c20264 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:31.190449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ae53c9e2-3434-44a5-92f0-877024c20264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:31.219224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:31.324378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 206/431] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:31.326207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53c9e2-3434-44a5-92f0-877024c20264 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:31.390055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 226/432] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 09:15:31.397586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85d82d-357c-48ac-9774-6d2ac61556a1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:31.504670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:31.509255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53c9e2-3434-44a5-92f0-877024c20264 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:31.559269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6332d1-cc5a-4bf9-9abe-8f687fa96ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:31.561121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6332d1-cc5a-4bf9-9abe-8f687fa96ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:31.561708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6332d1-cc5a-4bf9-9abe-8f687fa96ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:31.562124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6332d1-cc5a-4bf9-9abe-8f687fa96ed6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:31.573623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85d82d-357c-48ac-9774-6d2ac61556a1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:31.582243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6332d1-cc5a-4bf9-9abe-8f687fa96ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:31.593265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-7e6332d1-cc5a-4bf9-9abe-8f687fa96ed6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:31.598836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:31.599869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 09:15:31.600653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:31.601281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 207/433] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:31.602450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6332d1-cc5a-4bf9-9abe-8f687fa96ed6 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:31.628180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803f355-e1fc-4274-bb7f-e428a6a006cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:31.629049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803f355-e1fc-4274-bb7f-e428a6a006cc None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:31.629399 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803f355-e1fc-4274-bb7f-e428a6a006cc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:31.629548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803f355-e1fc-4274-bb7f-e428a6a006cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:31.643168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803f355-e1fc-4274-bb7f-e428a6a006cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-e803f355-e1fc-4274-bb7f-e428a6a006cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:31.651927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:31.653969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 227/434] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:15:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:31.655866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803f355-e1fc-4274-bb7f-e428a6a006cc None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:31.780106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803f355-e1fc-4274-bb7f-e428a6a006cc None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:31.806969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6332d1-cc5a-4bf9-9abe-8f687fa96ed6 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:31.837182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5bca6-898d-4218-9ddd-cb23539128da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:31.838168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5bca6-898d-4218-9ddd-cb23539128da None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:31.838616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5bca6-898d-4218-9ddd-cb23539128da None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:31.839063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5bca6-898d-4218-9ddd-cb23539128da None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:31.840333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5bca6-898d-4218-9ddd-cb23539128da None admin] RBAC: Adding query filter params (name=member) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:31.840848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5bca6-898d-4218-9ddd-cb23539128da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:31.843571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5bca6-898d-4218-9ddd-cb23539128da None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:31.862306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 228/435] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:15:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:31.862306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5bca6-898d-4218-9ddd-cb23539128da None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:31.864947 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23006576-1ed9-45a9-8e8e-c17a5fe199d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:31.865836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23006576-1ed9-45a9-8e8e-c17a5fe199d8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:31.866187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23006576-1ed9-45a9-8e8e-c17a5fe199d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:31.866463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23006576-1ed9-45a9-8e8e-c17a5fe199d8 None admin] PATH_INFO: `/v3/projects/8c495a22d3f7484cb377da68dab0bf97` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:31.867992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23006576-1ed9-45a9-8e8e-c17a5fe199d8 None admin] RBAC: Authorizing `identity:get_project(project_id=8c495a22d3f7484cb377da68dab0bf97)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:31.869620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23006576-1ed9-45a9-8e8e-c17a5fe199d8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:31.871477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 208/436] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:31 2022] GET /identity/v3/projects/8c495a22d3f7484cb377da68dab0bf97 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:31.877204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23006576-1ed9-45a9-8e8e-c17a5fe199d8 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:31.910613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:31.921692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:32.003297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23006576-1ed9-45a9-8e8e-c17a5fe199d8 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:32.026809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5bca6-898d-4218-9ddd-cb23539128da None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:32.063627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a69604-5b5b-4eb5-838a-f77f4aa6155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:32.064749 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a69604-5b5b-4eb5-838a-f77f4aa6155f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:32.065257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a69604-5b5b-4eb5-838a-f77f4aa6155f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:32.065692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a69604-5b5b-4eb5-838a-f77f4aa6155f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:32.079699 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a2bbb-19ce-4a96-ba24-f70fabf59671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:32.080499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a2bbb-19ce-4a96-ba24-f70fabf59671 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:32.080977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a2bbb-19ce-4a96-ba24-f70fabf59671 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:32.081309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a2bbb-19ce-4a96-ba24-f70fabf59671 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:32.082690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a2bbb-19ce-4a96-ba24-f70fabf59671 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:32.083060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a2bbb-19ce-4a96-ba24-f70fabf59671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:32.084490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a2bbb-19ce-4a96-ba24-f70fabf59671 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:32.099625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 229/437] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:32.099625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a2bbb-19ce-4a96-ba24-f70fabf59671 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:32.202002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a69604-5b5b-4eb5-838a-f77f4aa6155f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:32.204904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a69604-5b5b-4eb5-838a-f77f4aa6155f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:32.256055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:32.256997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:32.261394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a2bbb-19ce-4a96-ba24-f70fabf59671 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:32.321343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81a241-e909-430c-8072-1bcba6d4c08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:32.322542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81a241-e909-430c-8072-1bcba6d4c08d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:32.323039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81a241-e909-430c-8072-1bcba6d4c08d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:32.323421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81a241-e909-430c-8072-1bcba6d4c08d None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:32.325376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81a241-e909-430c-8072-1bcba6d4c08d None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:32.327627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81a241-e909-430c-8072-1bcba6d4c08d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:32.330530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 230/438] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:32 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:32.338119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81a241-e909-430c-8072-1bcba6d4c08d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:32.407279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 209/439] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:32.409562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a69604-5b5b-4eb5-838a-f77f4aa6155f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:32.498426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81a241-e909-430c-8072-1bcba6d4c08d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:32.557214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dbb8b-c7f1-4b44-9db3-66f0c090248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:32.558127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dbb8b-c7f1-4b44-9db3-66f0c090248c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:32.558551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dbb8b-c7f1-4b44-9db3-66f0c090248c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:32.559003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2dbb8b-c7f1-4b44-9db3-66f0c090248c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:32.578046 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a69604-5b5b-4eb5-838a-f77f4aa6155f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:32.619112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:32.622480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:32.642508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1fb49-7826-4629-9f88-797ada7f6c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:32.643750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1fb49-7826-4629-9f88-797ada7f6c71 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:32.650442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1fb49-7826-4629-9f88-797ada7f6c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:32.650442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1fb49-7826-4629-9f88-797ada7f6c71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:32.650442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1fb49-7826-4629-9f88-797ada7f6c71 None admin] RBAC: Adding query filter params (group.id=129eb944ca1047a1ac4820bd64d84ae8, role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=8c495a22d3f7484cb377da68dab0bf97) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:32.650442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1fb49-7826-4629-9f88-797ada7f6c71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:32.650442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1fb49-7826-4629-9f88-797ada7f6c71 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:32.667952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 210/440] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:32 2022] GET /identity/v3/role_assignments?group.id=129eb944ca1047a1ac4820bd64d84ae8&scope.project.id=8c495a22d3f7484cb377da68dab0bf97&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 611 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:32.667952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1fb49-7826-4629-9f88-797ada7f6c71 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:32.672713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dbb8b-c7f1-4b44-9db3-66f0c090248c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:32.674325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2dbb8b-c7f1-4b44-9db3-66f0c090248c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:32.830310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1fb49-7826-4629-9f88-797ada7f6c71 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:32.838236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 231/441] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:32.844315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dbb8b-c7f1-4b44-9db3-66f0c090248c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:32.896788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d035db-cb97-492c-be3e-33887214cc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:32.897715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d035db-cb97-492c-be3e-33887214cc78 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:32.898006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d035db-cb97-492c-be3e-33887214cc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:32.898288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d035db-cb97-492c-be3e-33887214cc78 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:32.900135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d035db-cb97-492c-be3e-33887214cc78 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:32.902003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d035db-cb97-492c-be3e-33887214cc78 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:32.904118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 211/442] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:32 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:32.913538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d035db-cb97-492c-be3e-33887214cc78 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:32.948636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:33.009952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2dbb8b-c7f1-4b44-9db3-66f0c090248c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:33.017429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:33.079606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316004a-a12e-4671-bb9c-e3b2fe30e1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:33.082582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316004a-a12e-4671-bb9c-e3b2fe30e1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:33.083073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316004a-a12e-4671-bb9c-e3b2fe30e1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:33.083491 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316004a-a12e-4671-bb9c-e3b2fe30e1e3 None admin] PATH_INFO: `/v3/projects/04cc0f746b104684b7b6e33d52a38126` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:33.084718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d035db-cb97-492c-be3e-33887214cc78 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:33.085865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316004a-a12e-4671-bb9c-e3b2fe30e1e3 None admin] RBAC: Authorizing `identity:get_project(project_id=04cc0f746b104684b7b6e33d52a38126)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:33.090435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316004a-a12e-4671-bb9c-e3b2fe30e1e3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:33.095646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 232/443] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:32 2022] GET /identity/v3/projects/04cc0f746b104684b7b6e33d52a38126 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:33.101916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316004a-a12e-4671-bb9c-e3b2fe30e1e3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:33.154270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec3fff-9f20-4940-8109-4ad1eccee734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:33.155522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec3fff-9f20-4940-8109-4ad1eccee734 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:33.155969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec3fff-9f20-4940-8109-4ad1eccee734 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:33.156379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec3fff-9f20-4940-8109-4ad1eccee734 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:33.158387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec3fff-9f20-4940-8109-4ad1eccee734 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:33.161251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec3fff-9f20-4940-8109-4ad1eccee734 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:33.163924 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 212/444] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:32 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:33.174568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec3fff-9f20-4940-8109-4ad1eccee734 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:33.265085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316004a-a12e-4671-bb9c-e3b2fe30e1e3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:33.316867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd68094-33f5-4f8b-9cb7-74fc21c65502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:33.317988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd68094-33f5-4f8b-9cb7-74fc21c65502 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:33.318295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd68094-33f5-4f8b-9cb7-74fc21c65502 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:33.318578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd68094-33f5-4f8b-9cb7-74fc21c65502 None admin] PATH_INFO: `/v3/groups/129eb944ca1047a1ac4820bd64d84ae8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:33.320000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd68094-33f5-4f8b-9cb7-74fc21c65502 None admin] RBAC: Authorizing `identity:get_group(group_id=129eb944ca1047a1ac4820bd64d84ae8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:33.321457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd68094-33f5-4f8b-9cb7-74fc21c65502 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:33.330935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 233/445] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:33 2022] GET /identity/v3/groups/129eb944ca1047a1ac4820bd64d84ae8 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:33.332592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd68094-33f5-4f8b-9cb7-74fc21c65502 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:33.340674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:33.345663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec3fff-9f20-4940-8109-4ad1eccee734 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:33.378321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:33.387624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913e70b-a8d9-457e-88db-f337fd83182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:33.388297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913e70b-a8d9-457e-88db-f337fd83182a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:33.388578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913e70b-a8d9-457e-88db-f337fd83182a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:33.388853 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913e70b-a8d9-457e-88db-f337fd83182a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:33.389693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913e70b-a8d9-457e-88db-f337fd83182a None admin] RBAC: Adding query filter params (group.id=129eb944ca1047a1ac4820bd64d84ae8, role.id=44cbd523412b44e3a591b3111fb50a71, scope.project.id=04cc0f746b104684b7b6e33d52a38126) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:33.390006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913e70b-a8d9-457e-88db-f337fd83182a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:33.391463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913e70b-a8d9-457e-88db-f337fd83182a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:33.400925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 213/446] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:33 2022] GET /identity/v3/role_assignments?group.id=129eb944ca1047a1ac4820bd64d84ae8&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=44cbd523412b44e3a591b3111fb50a71 => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:33.475101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd68094-33f5-4f8b-9cb7-74fc21c65502 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:33.522363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc51d9-9f0c-4b5d-8ac9-9f5146686b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:33.523374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc51d9-9f0c-4b5d-8ac9-9f5146686b81 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:33.523901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc51d9-9f0c-4b5d-8ac9-9f5146686b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:33.524420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc51d9-9f0c-4b5d-8ac9-9f5146686b81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:33.525735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc51d9-9f0c-4b5d-8ac9-9f5146686b81 None admin] RBAC: Adding query filter params (group.id=129eb944ca1047a1ac4820bd64d84ae8, role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=04cc0f746b104684b7b6e33d52a38126) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:33.526364 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc51d9-9f0c-4b5d-8ac9-9f5146686b81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:33.528076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc51d9-9f0c-4b5d-8ac9-9f5146686b81 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:33.543558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 234/447] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:33 2022] GET /identity/v3/role_assignments?group.id=129eb944ca1047a1ac4820bd64d84ae8&scope.project.id=04cc0f746b104684b7b6e33d52a38126&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:33.662446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573fb17-d0a4-49e8-84d5-f1e09a70689a None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:33.662967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573fb17-d0a4-49e8-84d5-f1e09a70689a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:33.663074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573fb17-d0a4-49e8-84d5-f1e09a70689a None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:33.664440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 214/448] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:33.670267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f32730-6dc6-46dd-b4cb-8373ddf7b694 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:33.670593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f32730-6dc6-46dd-b4cb-8373ddf7b694 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:33.670836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f32730-6dc6-46dd-b4cb-8373ddf7b694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:33.689968 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-18f32730-6dc6-46dd-b4cb-8373ddf7b694 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:33.691750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-18f32730-6dc6-46dd-b4cb-8373ddf7b694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:33.761663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:33.767485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 235/449] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:33.791656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573fb17-d0a4-49e8-84d5-f1e09a70689a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:33.914624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573fb17-d0a4-49e8-84d5-f1e09a70689a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:33.962607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3e5a2-a441-40b6-93f6-b85549a3d90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:33.963611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3e5a2-a441-40b6-93f6-b85549a3d90e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:33.963936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3e5a2-a441-40b6-93f6-b85549a3d90e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:33.964302 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3e5a2-a441-40b6-93f6-b85549a3d90e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:33.978724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3e5a2-a441-40b6-93f6-b85549a3d90e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-b5c3e5a2-a441-40b6-93f6-b85549a3d90e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:33.989970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:33.991823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:33.993367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 215/450] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:15:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:33.997437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f32730-6dc6-46dd-b4cb-8373ddf7b694 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:34.108874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f32730-6dc6-46dd-b4cb-8373ddf7b694 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:34.150251 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f40c5-adfe-4d26-9757-a915c3c349db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:34.150945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f40c5-adfe-4d26-9757-a915c3c349db None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:34.151233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f40c5-adfe-4d26-9757-a915c3c349db None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:34.151513 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f40c5-adfe-4d26-9757-a915c3c349db None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:34.152309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f40c5-adfe-4d26-9757-a915c3c349db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:34.152619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f40c5-adfe-4d26-9757-a915c3c349db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:34.154016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f40c5-adfe-4d26-9757-a915c3c349db None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:34.163952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 236/451] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:34.169860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3e5a2-a441-40b6-93f6-b85549a3d90e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:34.225756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:34.284293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3e5a2-a441-40b6-93f6-b85549a3d90e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:34.322108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962fbfa-170d-454a-bc68-b421898bbea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:34.322774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962fbfa-170d-454a-bc68-b421898bbea3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:34.323055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962fbfa-170d-454a-bc68-b421898bbea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:34.323334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962fbfa-170d-454a-bc68-b421898bbea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:34.400763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962fbfa-170d-454a-bc68-b421898bbea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:34.402245 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962fbfa-170d-454a-bc68-b421898bbea3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:34.456381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:34.511986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 216/452] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:34.518505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f40c5-adfe-4d26-9757-a915c3c349db None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:34.593890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:34.642960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f40c5-adfe-4d26-9757-a915c3c349db None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:34.681812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbd234-645a-4ab3-acbb-1119a87332ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:34.682597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbd234-645a-4ab3-acbb-1119a87332ca None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:34.682873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbd234-645a-4ab3-acbb-1119a87332ca None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:34.683151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbd234-645a-4ab3-acbb-1119a87332ca None admin] PATH_INFO: `/v3/projects/d706bacf3a904a118b091c4d80341fee` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:34.684397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbd234-645a-4ab3-acbb-1119a87332ca None admin] RBAC: Authorizing `identity:get_project(project_id=d706bacf3a904a118b091c4d80341fee)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:34.685847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbd234-645a-4ab3-acbb-1119a87332ca None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:34.687628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 237/453] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:15:34 2022] GET /identity/v3/projects/d706bacf3a904a118b091c4d80341fee => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:34.693641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962fbfa-170d-454a-bc68-b421898bbea3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:34.805854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962fbfa-170d-454a-bc68-b421898bbea3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:34.844127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647db3-3db3-460f-b42b-f354318ea8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:34.845004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647db3-3db3-460f-b42b-f354318ea8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:34.845284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647db3-3db3-460f-b42b-f354318ea8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:34.845560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647db3-3db3-460f-b42b-f354318ea8c8 None admin] PATH_INFO: `/v3/groups/43a7ae9398ab4e78a9ab45fc05d930e2` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:34.846813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647db3-3db3-460f-b42b-f354318ea8c8 None admin] RBAC: Authorizing `identity:get_group(group_id=43a7ae9398ab4e78a9ab45fc05d930e2)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:34.848916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647db3-3db3-460f-b42b-f354318ea8c8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:34.850624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 217/454] 10.222.0.64 () {56 vars in 1180 bytes} [Mon Jun 27 09:15:34 2022] GET /identity/v3/groups/43a7ae9398ab4e78a9ab45fc05d930e2 => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:34.856866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbd234-645a-4ab3-acbb-1119a87332ca None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:34.981111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbd234-645a-4ab3-acbb-1119a87332ca None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:35.011867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:35.020938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d87a39-b0d6-47d7-aaf2-8a1051196bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:35.026899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d87a39-b0d6-47d7-aaf2-8a1051196bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:35.026899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d87a39-b0d6-47d7-aaf2-8a1051196bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:35.026899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d87a39-b0d6-47d7-aaf2-8a1051196bf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:35.026899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d87a39-b0d6-47d7-aaf2-8a1051196bf1 None admin] RBAC: Adding query filter params (group.id=43a7ae9398ab4e78a9ab45fc05d930e2, role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=d706bacf3a904a118b091c4d80341fee) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:35.026899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d87a39-b0d6-47d7-aaf2-8a1051196bf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:35.026899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d87a39-b0d6-47d7-aaf2-8a1051196bf1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:35.032660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 238/455] 10.222.0.64 () {56 vars in 1376 bytes} [Mon Jun 27 09:15:34 2022] GET /identity/v3/role_assignments?group.id=43a7ae9398ab4e78a9ab45fc05d930e2&scope.project.id=d706bacf3a904a118b091c4d80341fee&role.id=03b724e03f704f2ab338b64025a3db6d => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:38.400304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf301435-da06-4c12-8f38-e75eaa870c31 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:38.401361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf301435-da06-4c12-8f38-e75eaa870c31 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:38.401868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf301435-da06-4c12-8f38-e75eaa870c31 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:38.403919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 218/456] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:38.410927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681231d-c088-410d-9909-7980134ddbbe None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:38.411975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681231d-c088-410d-9909-7980134ddbbe None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:38.412482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681231d-c088-410d-9909-7980134ddbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:38.436714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2681231d-c088-410d-9909-7980134ddbbe None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:38.438308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2681231d-c088-410d-9909-7980134ddbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:38.531704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 239/457] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:38.550362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf301435-da06-4c12-8f38-e75eaa870c31 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:38.564099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e32f96-77f2-4a1b-ac0c-9c4228a42936 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:38.564424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e32f96-77f2-4a1b-ac0c-9c4228a42936 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:38.564672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e32f96-77f2-4a1b-ac0c-9c4228a42936 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:38.567137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 240/458] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:38.568933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e7b36-cca0-4b03-b84a-86e71d81f80f None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:38.569230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e7b36-cca0-4b03-b84a-86e71d81f80f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:38.569511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e7b36-cca0-4b03-b84a-86e71d81f80f None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:38.571195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 241/459] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:38.572724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292cb53-3ac8-443e-ae39-f253d8483bae None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:38.573014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292cb53-3ac8-443e-ae39-f253d8483bae None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:38.573295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292cb53-3ac8-443e-ae39-f253d8483bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:38.579904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:38.595280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e292cb53-3ac8-443e-ae39-f253d8483bae None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:38.596559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e292cb53-3ac8-443e-ae39-f253d8483bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:38.685799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 242/460] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:38.687441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84622663-b1f7-48f2-b4d8-b16784fcf80e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:38.687914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84622663-b1f7-48f2-b4d8-b16784fcf80e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:38.688334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84622663-b1f7-48f2-b4d8-b16784fcf80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:38.708867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-84622663-b1f7-48f2-b4d8-b16784fcf80e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:38.710507 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-84622663-b1f7-48f2-b4d8-b16784fcf80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:38.711296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf301435-da06-4c12-8f38-e75eaa870c31 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:38.759320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87caa8-79f1-4d7d-9528-e5448cd17d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:38.760586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87caa8-79f1-4d7d-9528-e5448cd17d6d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:38.761082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87caa8-79f1-4d7d-9528-e5448cd17d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:38.761557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87caa8-79f1-4d7d-9528-e5448cd17d6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:38.773996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:38.775804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87caa8-79f1-4d7d-9528-e5448cd17d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:38.778312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87caa8-79f1-4d7d-9528-e5448cd17d6d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-ee87caa8-79f1-4d7d-9528-e5448cd17d6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:38.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:38.799636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:38.800581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:38.800581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:38.800581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:38.800581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:38.801015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 243/461] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:38.802357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84622663-b1f7-48f2-b4d8-b16784fcf80e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:38.802998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 219/462] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:38.809051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87caa8-79f1-4d7d-9528-e5448cd17d6d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:38.913021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:38.954858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87caa8-79f1-4d7d-9528-e5448cd17d6d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:38.956190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84622663-b1f7-48f2-b4d8-b16784fcf80e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:39.001180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe136546-395b-4d27-8f5f-7d373ac9d469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:39.002559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe136546-395b-4d27-8f5f-7d373ac9d469 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.004066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe136546-395b-4d27-8f5f-7d373ac9d469 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.004590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe136546-395b-4d27-8f5f-7d373ac9d469 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.005927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe136546-395b-4d27-8f5f-7d373ac9d469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:39.006471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe136546-395b-4d27-8f5f-7d373ac9d469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:39.009075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe136546-395b-4d27-8f5f-7d373ac9d469 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:39.013050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32954f90-f27e-4c38-84c8-68f2a4fb37b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:39.014218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954f90-f27e-4c38-84c8-68f2a4fb37b9 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.014719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954f90-f27e-4c38-84c8-68f2a4fb37b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.015227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32954f90-f27e-4c38-84c8-68f2a4fb37b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.029780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 220/463] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:39.029780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe136546-395b-4d27-8f5f-7d373ac9d469 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:39.033274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32954f90-f27e-4c38-84c8-68f2a4fb37b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:39.035005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32954f90-f27e-4c38-84c8-68f2a4fb37b9 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-32954f90-f27e-4c38-84c8-68f2a4fb37b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:39.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:39.055658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:39.056516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:39.056516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:39.056516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:39.056516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:39.056516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 244/464] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:39.057184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92364bee-7554-4d54-959b-6d16176ea15f None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.057633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92364bee-7554-4d54-959b-6d16176ea15f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.058048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92364bee-7554-4d54-959b-6d16176ea15f None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.059728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 245/465] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:39.062154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92364bee-7554-4d54-959b-6d16176ea15f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:39.125958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:39.182914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe136546-395b-4d27-8f5f-7d373ac9d469 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:39.229705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:39.236354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f17684-0b94-47b0-8cbe-88ab3e62754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:39.237312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17684-0b94-47b0-8cbe-88ab3e62754d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.237697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17684-0b94-47b0-8cbe-88ab3e62754d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.238112 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f17684-0b94-47b0-8cbe-88ab3e62754d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.245636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92364bee-7554-4d54-959b-6d16176ea15f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:39.249594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f17684-0b94-47b0-8cbe-88ab3e62754d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:39.251370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f17684-0b94-47b0-8cbe-88ab3e62754d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-40f17684-0b94-47b0-8cbe-88ab3e62754d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:39.278247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:39.279108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:39.280083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:39.280083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:39.280083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:39.280083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:39.282317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 221/466] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:39.284433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db1d85-3ef3-4e8b-9b3f-308f284bee56 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.285026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db1d85-3ef3-4e8b-9b3f-308f284bee56 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.285516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db1d85-3ef3-4e8b-9b3f-308f284bee56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.291194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a65aa4-430c-4060-a561-aa801f442299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:39.291915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a65aa4-430c-4060-a561-aa801f442299 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.292197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a65aa4-430c-4060-a561-aa801f442299 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.292476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a65aa4-430c-4060-a561-aa801f442299 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.293242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a65aa4-430c-4060-a561-aa801f442299 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:39.294741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a65aa4-430c-4060-a561-aa801f442299 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:39.313144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e4db1d85-3ef3-4e8b-9b3f-308f284bee56 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:39.314405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e4db1d85-3ef3-4e8b-9b3f-308f284bee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:39.409257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 222/467] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:39.409257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db1d85-3ef3-4e8b-9b3f-308f284bee56 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:39.433022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 246/468] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:15:39 2022] POST /identity/v3/users => generated 248 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:15:39.434821 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a65aa4-430c-4060-a561-aa801f442299 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:39.568964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db1d85-3ef3-4e8b-9b3f-308f284bee56 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:39.582778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:39.588155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:39.596615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a65aa4-430c-4060-a561-aa801f442299 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:39.613440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a80763-d822-47d6-9025-a05e49f7ccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:39.614119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a80763-d822-47d6-9025-a05e49f7ccab None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.614413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a80763-d822-47d6-9025-a05e49f7ccab None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.614839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a80763-d822-47d6-9025-a05e49f7ccab None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.615611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a80763-d822-47d6-9025-a05e49f7ccab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:39.615922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a80763-d822-47d6-9025-a05e49f7ccab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:39.617502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a80763-d822-47d6-9025-a05e49f7ccab None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:39.630246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 223/469] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:39.634131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a80763-d822-47d6-9025-a05e49f7ccab None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:39.643097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f49c32-4c7e-4dfd-8b62-31aa04c8d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:39.644633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f49c32-4c7e-4dfd-8b62-31aa04c8d44a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.644921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f49c32-4c7e-4dfd-8b62-31aa04c8d44a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.645198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f49c32-4c7e-4dfd-8b62-31aa04c8d44a None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.645947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f49c32-4c7e-4dfd-8b62-31aa04c8d44a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:39.646254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f49c32-4c7e-4dfd-8b62-31aa04c8d44a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:39.647750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f49c32-4c7e-4dfd-8b62-31aa04c8d44a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:39.670398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 247/470] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:39.672760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb637f4-1b26-484d-b68e-a16118bf8835 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.673055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb637f4-1b26-484d-b68e-a16118bf8835 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.673338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb637f4-1b26-484d-b68e-a16118bf8835 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.674727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 248/471] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:39.675451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb637f4-1b26-484d-b68e-a16118bf8835 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:39.763054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a80763-d822-47d6-9025-a05e49f7ccab None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:39.801130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad644c-7364-49f8-81fd-093ab32a8278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:39.801857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad644c-7364-49f8-81fd-093ab32a8278 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.802143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad644c-7364-49f8-81fd-093ab32a8278 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.802426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad644c-7364-49f8-81fd-093ab32a8278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.814399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad644c-7364-49f8-81fd-093ab32a8278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:39.816399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad644c-7364-49f8-81fd-093ab32a8278 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:39.823487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb637f4-1b26-484d-b68e-a16118bf8835 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-78ad644c-7364-49f8-81fd-093ab32a8278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:39.835751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:39.836877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:39.837720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:39.837720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:39.837720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:39.837720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:39.837720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 224/472] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:39.838007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad644c-7364-49f8-81fd-093ab32a8278 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:39.866490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91206fdf-df27-4b37-b673-c539c4e734db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:39.866851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91206fdf-df27-4b37-b673-c539c4e734db None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.867136 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91206fdf-df27-4b37-b673-c539c4e734db None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.867418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91206fdf-df27-4b37-b673-c539c4e734db None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.867775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:39.868452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91206fdf-df27-4b37-b673-c539c4e734db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:39.870082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91206fdf-df27-4b37-b673-c539c4e734db None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:39.960953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad644c-7364-49f8-81fd-093ab32a8278 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:39.977089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 249/473] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:15:39 2022] POST /identity/v3/users => generated 246 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:15:39.978463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31810e3e-fcd7-4e69-bf63-088486c3d2c6 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:39.978762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31810e3e-fcd7-4e69-bf63-088486c3d2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:39.979181 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31810e3e-fcd7-4e69-bf63-088486c3d2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:39.995972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:40.010874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-31810e3e-fcd7-4e69-bf63-088486c3d2c6 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:40.015244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-31810e3e-fcd7-4e69-bf63-088486c3d2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:40.020716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63920269-5412-4d6a-acba-b2acf6a2a786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:40.022190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63920269-5412-4d6a-acba-b2acf6a2a786 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:40.022190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63920269-5412-4d6a-acba-b2acf6a2a786 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:40.022190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63920269-5412-4d6a-acba-b2acf6a2a786 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:40.023185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63920269-5412-4d6a-acba-b2acf6a2a786 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:40.025354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63920269-5412-4d6a-acba-b2acf6a2a786 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:40.139411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 250/474] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:40.139411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31810e3e-fcd7-4e69-bf63-088486c3d2c6 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:40.154026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 225/475] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:15:39 2022] POST /identity/v3/users => generated 249 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:15:40.197518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63920269-5412-4d6a-acba-b2acf6a2a786 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:40.343762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31810e3e-fcd7-4e69-bf63-088486c3d2c6 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:40.364869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:40.402744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63920269-5412-4d6a-acba-b2acf6a2a786 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:40.412587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd711e9-2cf9-44fb-a3f2-871293d9d76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:40.414779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd711e9-2cf9-44fb-a3f2-871293d9d76f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:40.415803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd711e9-2cf9-44fb-a3f2-871293d9d76f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:40.416568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd711e9-2cf9-44fb-a3f2-871293d9d76f None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:40.418341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd711e9-2cf9-44fb-a3f2-871293d9d76f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:40.419301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd711e9-2cf9-44fb-a3f2-871293d9d76f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:40.422307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd711e9-2cf9-44fb-a3f2-871293d9d76f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:40.427482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:40.446575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:40.447307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:40.447837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:40.447989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:40.449186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:40.449625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 251/476] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:40.450548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:40.458320 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd711e9-2cf9-44fb-a3f2-871293d9d76f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:40.506206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.sql.core [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 09:15:40.506206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 09:15:40.506206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [parameters: {'id': 'd8059f399adb4b838fd97e62949058e8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 09:15:40.506206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80344) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application conn.query(q) Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result.read() Jun 27 09:15:40.542515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return ref.to_dict() Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 09:15:40.543461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application next(self.gen) Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application yield resource Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application next(self.gen) Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise self.value Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application session.commit() Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 09:15:40.544454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.session.flush() Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application compat.raise_( Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 09:15:40.545356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 09:15:40.546269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application conn.query(q) Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result.read() Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 09:15:40.547120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [parameters: {'id': 'd8059f399adb4b838fd97e62949058e8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:40.547963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 09:15:40.548758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:40.554481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 226/477] 10.222.0.64 () {60 vars in 1131 bytes} [Mon Jun 27 09:15:40 2022] POST /identity/v3/roles => generated 144 bytes in 357 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:40.564820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:40.662847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd711e9-2cf9-44fb-a3f2-871293d9d76f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:40.711983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90f8b1-a27d-402b-a329-b5fca5a713da None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:40.730612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c1665-9f4f-415e-bd43-405719f843fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:40.731859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c1665-9f4f-415e-bd43-405719f843fa None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:40.732657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c1665-9f4f-415e-bd43-405719f843fa None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:40.732779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c1665-9f4f-415e-bd43-405719f843fa None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:40.734303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c1665-9f4f-415e-bd43-405719f843fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:40.737174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c1665-9f4f-415e-bd43-405719f843fa None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:40.766921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e57fa4-51b0-4f11-87bc-37773181ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:40.767807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e57fa4-51b0-4f11-87bc-37773181ec42 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:40.768190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e57fa4-51b0-4f11-87bc-37773181ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:40.768690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e57fa4-51b0-4f11-87bc-37773181ec42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:40.781954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e57fa4-51b0-4f11-87bc-37773181ec42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-25e57fa4-51b0-4f11-87bc-37773181ec42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:40.791274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:40.792472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:40.793371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 227/478] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:15:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:40.804251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e57fa4-51b0-4f11-87bc-37773181ec42 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:40.815109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:40.884940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 252/479] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:15:40 2022] POST /identity/v3/users => generated 248 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:15:41.029887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e57fa4-51b0-4f11-87bc-37773181ec42 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:41.103407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fa03c-cad4-4a4b-acd6-d94de3248452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:41.105368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fa03c-cad4-4a4b-acd6-d94de3248452 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:41.106243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fa03c-cad4-4a4b-acd6-d94de3248452 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:41.107082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fa03c-cad4-4a4b-acd6-d94de3248452 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:41.108878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fa03c-cad4-4a4b-acd6-d94de3248452 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:41.109722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fa03c-cad4-4a4b-acd6-d94de3248452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:41.112999 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fa03c-cad4-4a4b-acd6-d94de3248452 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:41.131192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 228/480] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:41.772667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0dbc8-63ac-405d-a310-a52e81b62a97 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:41.773192 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0dbc8-63ac-405d-a310-a52e81b62a97 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:41.773273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0dbc8-63ac-405d-a310-a52e81b62a97 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:41.775800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 253/481] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:41.782433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cde608-220f-493e-968c-669130f40e36 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:41.782943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cde608-220f-493e-968c-669130f40e36 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:41.783268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cde608-220f-493e-968c-669130f40e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:41.821542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b4cde608-220f-493e-968c-669130f40e36 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:41.823054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b4cde608-220f-493e-968c-669130f40e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:41.969491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:41.988468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 229/482] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:42.030443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0dbc8-63ac-405d-a310-a52e81b62a97 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:42.151516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:42.258625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0dbc8-63ac-405d-a310-a52e81b62a97 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:42.341570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70082133-6d96-43c7-9c01-a9eebcc027ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:42.342832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70082133-6d96-43c7-9c01-a9eebcc027ff None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:42.343385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70082133-6d96-43c7-9c01-a9eebcc027ff None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:42.343899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70082133-6d96-43c7-9c01-a9eebcc027ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:42.364874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70082133-6d96-43c7-9c01-a9eebcc027ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-70082133-6d96-43c7-9c01-a9eebcc027ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:42.381287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:42.382247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:42.383052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:42.384270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 254/483] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:15:42.397517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cde608-220f-493e-968c-669130f40e36 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:42.636048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cde608-220f-493e-968c-669130f40e36 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:42.726118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e1bd1-9949-4057-9f60-6c42307c932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:42.727452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e1bd1-9949-4057-9f60-6c42307c932e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:42.728003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e1bd1-9949-4057-9f60-6c42307c932e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:42.728511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e1bd1-9949-4057-9f60-6c42307c932e None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:42.730132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e1bd1-9949-4057-9f60-6c42307c932e None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:42.730701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e1bd1-9949-4057-9f60-6c42307c932e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:42.733632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e1bd1-9949-4057-9f60-6c42307c932e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:42.751342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 230/484] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:42.758412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70082133-6d96-43c7-9c01-a9eebcc027ff None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:42.942584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70082133-6d96-43c7-9c01-a9eebcc027ff None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:42.979646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:43.011328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9453bd-c6bb-447b-a481-1e129ba5ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:43.012560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9453bd-c6bb-447b-a481-1e129ba5ebd0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:43.013050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9453bd-c6bb-447b-a481-1e129ba5ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:43.013610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9453bd-c6bb-447b-a481-1e129ba5ebd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:43.027260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9453bd-c6bb-447b-a481-1e129ba5ebd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:43.033270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9453bd-c6bb-447b-a481-1e129ba5ebd0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-fb9453bd-c6bb-447b-a481-1e129ba5ebd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:43.058007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:43.058919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:43.059844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:43.059844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:43.059844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:43.059844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:43.061984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 255/485] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:43.071353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e1bd1-9949-4057-9f60-6c42307c932e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:43.180730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:43.246284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e1bd1-9949-4057-9f60-6c42307c932e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:43.306129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e67e4-778c-4979-b978-b118247cdaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:43.307187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e67e4-778c-4979-b978-b118247cdaad None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:43.307734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e67e4-778c-4979-b978-b118247cdaad None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:43.308223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e67e4-778c-4979-b978-b118247cdaad None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:43.309182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e67e4-778c-4979-b978-b118247cdaad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:43.309703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e67e4-778c-4979-b978-b118247cdaad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:43.311306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e67e4-778c-4979-b978-b118247cdaad None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:43.330679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b1742-2796-4516-826c-8f0321738eea None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:43.331355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b1742-2796-4516-826c-8f0321738eea None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:43.331885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b1742-2796-4516-826c-8f0321738eea None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:43.333873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 256/486] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:43.338714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 231/487] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:43.345042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecf95e-db46-47f1-bbbc-82259b50764e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:43.345575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecf95e-db46-47f1-bbbc-82259b50764e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:43.346060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecf95e-db46-47f1-bbbc-82259b50764e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:43.354032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e67e4-778c-4979-b978-b118247cdaad None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:43.382498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a9ecf95e-db46-47f1-bbbc-82259b50764e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:43.384262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a9ecf95e-db46-47f1-bbbc-82259b50764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:43.533879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e67e4-778c-4979-b978-b118247cdaad None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:43.556665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:43.582476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2abff6-cd76-4bd0-8a1d-cde8afd603a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:43.583371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2abff6-cd76-4bd0-8a1d-cde8afd603a4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:43.583688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2abff6-cd76-4bd0-8a1d-cde8afd603a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:43.583971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2abff6-cd76-4bd0-8a1d-cde8afd603a4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:43.601097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2abff6-cd76-4bd0-8a1d-cde8afd603a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:43.603997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2abff6-cd76-4bd0-8a1d-cde8afd603a4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:43.620284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 257/488] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-1d2abff6-cd76-4bd0-8a1d-cde8afd603a4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:43.649741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:43.651004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:43.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 232/489] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:15:43.656355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecf95e-db46-47f1-bbbc-82259b50764e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:43.657004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2abff6-cd76-4bd0-8a1d-cde8afd603a4 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:43.826543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:43.846434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2abff6-cd76-4bd0-8a1d-cde8afd603a4 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:43.854072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecf95e-db46-47f1-bbbc-82259b50764e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:43.885514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67019b02-26f3-44c0-b367-3ada0bfe1b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:43.886193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67019b02-26f3-44c0-b367-3ada0bfe1b9f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:43.886475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67019b02-26f3-44c0-b367-3ada0bfe1b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:43.886752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67019b02-26f3-44c0-b367-3ada0bfe1b9f None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:43.887677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67019b02-26f3-44c0-b367-3ada0bfe1b9f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:43.887994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67019b02-26f3-44c0-b367-3ada0bfe1b9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:43.889702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67019b02-26f3-44c0-b367-3ada0bfe1b9f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:43.895094 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685c2c7-9ba0-4ed6-8688-d520fd750f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:43.896077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685c2c7-9ba0-4ed6-8688-d520fd750f1f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:43.896485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685c2c7-9ba0-4ed6-8688-d520fd750f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:43.896860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685c2c7-9ba0-4ed6-8688-d520fd750f1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:43.908037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685c2c7-9ba0-4ed6-8688-d520fd750f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-4685c2c7-9ba0-4ed6-8688-d520fd750f1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:43.917060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:43.918891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:43.921023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 258/490] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:43.922686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f2bc1-9774-4ef4-b9dc-bebf43461d81 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:43.922686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f2bc1-9774-4ef4-b9dc-bebf43461d81 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:43.922686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f2bc1-9774-4ef4-b9dc-bebf43461d81 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:43.927045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 259/491] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:43.929459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f2bc1-9774-4ef4-b9dc-bebf43461d81 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:43.936268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 233/492] 10.222.0.64 () {56 vars in 1137 bytes} [Mon Jun 27 09:15:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:43.937614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588d8c1-e090-447a-8de1-b617b48e31ab None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:43.937909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588d8c1-e090-447a-8de1-b617b48e31ab None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:43.938185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588d8c1-e090-447a-8de1-b617b48e31ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:43.960835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e588d8c1-e090-447a-8de1-b617b48e31ab None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:43.962447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e588d8c1-e090-447a-8de1-b617b48e31ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:43.998721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:44.053988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 234/493] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:44.055226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588d8c1-e090-447a-8de1-b617b48e31ab None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:44.068624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f2bc1-9774-4ef4-b9dc-bebf43461d81 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:44.125377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6c105-ee72-46c6-85ed-b4d9447bf578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:44.126125 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6c105-ee72-46c6-85ed-b4d9447bf578 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.126421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6c105-ee72-46c6-85ed-b4d9447bf578 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.126705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6c105-ee72-46c6-85ed-b4d9447bf578 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.127619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6c105-ee72-46c6-85ed-b4d9447bf578 None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:44.127985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6c105-ee72-46c6-85ed-b4d9447bf578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:44.129598 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6c105-ee72-46c6-85ed-b4d9447bf578 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:44.139012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 260/494] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:44.140357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fab81-9cd9-4e5a-9a83-7c0655c6d423 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.140647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fab81-9cd9-4e5a-9a83-7c0655c6d423 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.140920 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fab81-9cd9-4e5a-9a83-7c0655c6d423 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.148134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 261/495] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:44.148134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fab81-9cd9-4e5a-9a83-7c0655c6d423 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:44.170068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:44.234257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588d8c1-e090-447a-8de1-b617b48e31ab None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:44.297079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26540077-3155-43a7-9f18-fccf6144d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:44.298484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26540077-3155-43a7-9f18-fccf6144d6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.298715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26540077-3155-43a7-9f18-fccf6144d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.299315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26540077-3155-43a7-9f18-fccf6144d6b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.307655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fab81-9cd9-4e5a-9a83-7c0655c6d423 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:44.339684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:44.352651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46d994-d9c4-453f-99b9-92ddf855b233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:44.353468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46d994-d9c4-453f-99b9-92ddf855b233 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.353748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46d994-d9c4-453f-99b9-92ddf855b233 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.354025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46d994-d9c4-453f-99b9-92ddf855b233 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.366380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46d994-d9c4-453f-99b9-92ddf855b233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:44.376561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26540077-3155-43a7-9f18-fccf6144d6b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:44.377970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26540077-3155-43a7-9f18-fccf6144d6b9 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-bd46d994-d9c4-453f-99b9-92ddf855b233 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:44.382980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:44.384195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:44.385355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:44.386061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 262/496] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:44.387785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7ab40-08d9-4887-bfdf-28c7bf7d4872 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.388308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7ab40-08d9-4887-bfdf-28c7bf7d4872 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.388824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7ab40-08d9-4887-bfdf-28c7bf7d4872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.411716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e9e7ab40-08d9-4887-bfdf-28c7bf7d4872 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:44.412944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e9e7ab40-08d9-4887-bfdf-28c7bf7d4872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:44.489243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 235/497] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:44.490116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26540077-3155-43a7-9f18-fccf6144d6b9 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:44.499596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 263/498] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:44.501483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1440-1d58-45f3-92ec-ebde174d8d12 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.501819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1440-1d58-45f3-92ec-ebde174d8d12 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.502120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c1440-1d58-45f3-92ec-ebde174d8d12 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.504041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 264/499] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:44.505051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c1440-1d58-45f3-92ec-ebde174d8d12 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:44.523177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:44.568853 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:44.610071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26540077-3155-43a7-9f18-fccf6144d6b9 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:44.630304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c1440-1d58-45f3-92ec-ebde174d8d12 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:44.646837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05184e-a29d-45c4-a57e-2002090eb034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:44.647670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05184e-a29d-45c4-a57e-2002090eb034 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.647952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05184e-a29d-45c4-a57e-2002090eb034 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.648230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05184e-a29d-45c4-a57e-2002090eb034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.659095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05184e-a29d-45c4-a57e-2002090eb034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:44.660606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05184e-a29d-45c4-a57e-2002090eb034 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-8e05184e-a29d-45c4-a57e-2002090eb034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:44.678389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:44.679284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:44.680253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:44.680253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:44.680253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:44.680253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:44.680253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 236/500] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:44.680696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05184e-a29d-45c4-a57e-2002090eb034 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:44.685580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0929e-8825-44b5-9b99-c924c0734243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:44.686275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0929e-8825-44b5-9b99-c924c0734243 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.686558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0929e-8825-44b5-9b99-c924c0734243 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.686825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0929e-8825-44b5-9b99-c924c0734243 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.687587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0929e-8825-44b5-9b99-c924c0734243 None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:44.687895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0929e-8825-44b5-9b99-c924c0734243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:44.689267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0929e-8825-44b5-9b99-c924c0734243 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:44.698112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 265/501] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:44.699390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bb5a4-44ba-4032-a004-b00763747a37 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.699708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bb5a4-44ba-4032-a004-b00763747a37 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.699986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bb5a4-44ba-4032-a004-b00763747a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.718906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-418bb5a4-44ba-4032-a004-b00763747a37 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:44.721087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-418bb5a4-44ba-4032-a004-b00763747a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:44.804584 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05184e-a29d-45c4-a57e-2002090eb034 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:44.807496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:44.813336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 266/502] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:44.814359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bb5a4-44ba-4032-a004-b00763747a37 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:44.837902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:44.855211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857557c-4fdd-433f-a55d-321d6cccba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:44.856050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857557c-4fdd-433f-a55d-321d6cccba23 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.856328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857557c-4fdd-433f-a55d-321d6cccba23 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.856608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857557c-4fdd-433f-a55d-321d6cccba23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.874857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857557c-4fdd-433f-a55d-321d6cccba23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:44.876412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857557c-4fdd-433f-a55d-321d6cccba23 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0857557c-4fdd-433f-a55d-321d6cccba23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:44.895479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:44.896421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:44.897243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:44.897243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:44.897243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:44.897243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:44.897243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 237/503] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:44.897507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857557c-4fdd-433f-a55d-321d6cccba23 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:44.937093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bb5a4-44ba-4032-a004-b00763747a37 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:44.984972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203bdee-5a25-44a3-b40d-f88b276e56a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:44.985829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203bdee-5a25-44a3-b40d-f88b276e56a2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:44.986111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203bdee-5a25-44a3-b40d-f88b276e56a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:44.986387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203bdee-5a25-44a3-b40d-f88b276e56a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:44.998167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203bdee-5a25-44a3-b40d-f88b276e56a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.000130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203bdee-5a25-44a3-b40d-f88b276e56a2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.011058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857557c-4fdd-433f-a55d-321d6cccba23 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-d203bdee-5a25-44a3-b40d-f88b276e56a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:45.027425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:45.028438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:45.029260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:45.029260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:45.029260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:45.029260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:45.029260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 267/504] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:45.029522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203bdee-5a25-44a3-b40d-f88b276e56a2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.054687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071141e6-358f-452d-8461-0d7e9ad163bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.055358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071141e6-358f-452d-8461-0d7e9ad163bd None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.055679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071141e6-358f-452d-8461-0d7e9ad163bd None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.055963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071141e6-358f-452d-8461-0d7e9ad163bd None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.056704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071141e6-358f-452d-8461-0d7e9ad163bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:45.057018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071141e6-358f-452d-8461-0d7e9ad163bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.058432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071141e6-358f-452d-8461-0d7e9ad163bd None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.070638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 238/505] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:45.071495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071141e6-358f-452d-8461-0d7e9ad163bd None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.083011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:45.105707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:45.136677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203bdee-5a25-44a3-b40d-f88b276e56a2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.173007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bd1af-b4ad-4b94-b58f-a29b764e9dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.173714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bd1af-b4ad-4b94-b58f-a29b764e9dcf None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.173994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bd1af-b4ad-4b94-b58f-a29b764e9dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.174268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bd1af-b4ad-4b94-b58f-a29b764e9dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.182134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071141e6-358f-452d-8461-0d7e9ad163bd None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.185859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96bd1af-b4ad-4b94-b58f-a29b764e9dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.187317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96bd1af-b4ad-4b94-b58f-a29b764e9dcf None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-a96bd1af-b4ad-4b94-b58f-a29b764e9dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:45.205732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:45.206647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:45.207512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:45.207512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:45.207512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:45.207512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:45.207512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 268/506] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:45.207870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bd1af-b4ad-4b94-b58f-a29b764e9dcf None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.221004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24aeb8-1c74-45e4-b345-782e77c2ff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.221809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24aeb8-1c74-45e4-b345-782e77c2ff5c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.222583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24aeb8-1c74-45e4-b345-782e77c2ff5c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.222854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24aeb8-1c74-45e4-b345-782e77c2ff5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.235194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24aeb8-1c74-45e4-b345-782e77c2ff5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-be24aeb8-1c74-45e4-b345-782e77c2ff5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:45.244159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:45.245506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 239/507] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:45.246333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24aeb8-1c74-45e4-b345-782e77c2ff5c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.322524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bd1af-b4ad-4b94-b58f-a29b764e9dcf None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.324401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:45.352990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:45.364341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24aeb8-1c74-45e4-b345-782e77c2ff5c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.383513 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44473c3d-ffce-45cf-9df0-2c6c5c3e5bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.384529 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44473c3d-ffce-45cf-9df0-2c6c5c3e5bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.384931 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44473c3d-ffce-45cf-9df0-2c6c5c3e5bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.385295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44473c3d-ffce-45cf-9df0-2c6c5c3e5bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.386163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44473c3d-ffce-45cf-9df0-2c6c5c3e5bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:45.386560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44473c3d-ffce-45cf-9df0-2c6c5c3e5bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.388486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44473c3d-ffce-45cf-9df0-2c6c5c3e5bf3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.401658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 269/508] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:45.406248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44473c3d-ffce-45cf-9df0-2c6c5c3e5bf3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.406412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eec0d5-e2c7-4bfb-9818-b228f96cef65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.406412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eec0d5-e2c7-4bfb-9818-b228f96cef65 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.406412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eec0d5-e2c7-4bfb-9818-b228f96cef65 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.406412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eec0d5-e2c7-4bfb-9818-b228f96cef65 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.406737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eec0d5-e2c7-4bfb-9818-b228f96cef65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:45.406737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eec0d5-e2c7-4bfb-9818-b228f96cef65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.407409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eec0d5-e2c7-4bfb-9818-b228f96cef65 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.420763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 240/509] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:45.421676 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eec0d5-e2c7-4bfb-9818-b228f96cef65 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.506101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44473c3d-ffce-45cf-9df0-2c6c5c3e5bf3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.525626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eec0d5-e2c7-4bfb-9818-b228f96cef65 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.542121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bc937-abe9-4022-8990-734313deb40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.542928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc937-abe9-4022-8990-734313deb40c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.543210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc937-abe9-4022-8990-734313deb40c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.543486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc937-abe9-4022-8990-734313deb40c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.551704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bc937-abe9-4022-8990-734313deb40c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.553190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bc937-abe9-4022-8990-734313deb40c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.561983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075e95e-5d17-49ac-bc54-06fe8cea4192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.562666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075e95e-5d17-49ac-bc54-06fe8cea4192 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.562932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075e95e-5d17-49ac-bc54-06fe8cea4192 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.563201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075e95e-5d17-49ac-bc54-06fe8cea4192 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.563965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075e95e-5d17-49ac-bc54-06fe8cea4192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:45.564502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075e95e-5d17-49ac-bc54-06fe8cea4192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.565690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075e95e-5d17-49ac-bc54-06fe8cea4192 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-541bc937-abe9-4022-8990-734313deb40c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:45.569053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:45.570135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 270/510] 10.222.0.64 () {56 vars in 1093 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:45.571214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bc937-abe9-4022-8990-734313deb40c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.583501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 241/511] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:45.584401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075e95e-5d17-49ac-bc54-06fe8cea4192 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.599471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:45.603584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:45.685355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bc937-abe9-4022-8990-734313deb40c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.695696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075e95e-5d17-49ac-bc54-06fe8cea4192 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.721602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241334b-6f25-4e5b-a715-e3e9c27c4b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.722304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241334b-6f25-4e5b-a715-e3e9c27c4b0e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.722588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241334b-6f25-4e5b-a715-e3e9c27c4b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.722864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241334b-6f25-4e5b-a715-e3e9c27c4b0e None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.723663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241334b-6f25-4e5b-a715-e3e9c27c4b0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:45.723974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241334b-6f25-4e5b-a715-e3e9c27c4b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.725385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241334b-6f25-4e5b-a715-e3e9c27c4b0e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.733836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66021-98ce-4a44-b23c-34b6c0ce766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.734031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 271/512] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:45.734653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66021-98ce-4a44-b23c-34b6c0ce766f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.734901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241334b-6f25-4e5b-a715-e3e9c27c4b0e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.735003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66021-98ce-4a44-b23c-34b6c0ce766f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.735227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e66021-98ce-4a44-b23c-34b6c0ce766f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.744934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66021-98ce-4a44-b23c-34b6c0ce766f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.746405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e66021-98ce-4a44-b23c-34b6c0ce766f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-33e66021-98ce-4a44-b23c-34b6c0ce766f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:45.761790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:45.762858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 242/513] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:45.763993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66021-98ce-4a44-b23c-34b6c0ce766f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.827842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:45.842591 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:45.845384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241334b-6f25-4e5b-a715-e3e9c27c4b0e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.875508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e66021-98ce-4a44-b23c-34b6c0ce766f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:45.881546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a0767-e15d-4bcb-afec-2e26b9d73def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.882228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a0767-e15d-4bcb-afec-2e26b9d73def None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.882511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a0767-e15d-4bcb-afec-2e26b9d73def None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.882791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a0767-e15d-4bcb-afec-2e26b9d73def None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.883511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a0767-e15d-4bcb-afec-2e26b9d73def None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.885002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a0767-e15d-4bcb-afec-2e26b9d73def None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.913082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d83d2c-ab46-4c69-b208-34103ebdc3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:45.913763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d83d2c-ab46-4c69-b208-34103ebdc3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:45.914131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d83d2c-ab46-4c69-b208-34103ebdc3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:45.914404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d83d2c-ab46-4c69-b208-34103ebdc3c4 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:45.915247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d83d2c-ab46-4c69-b208-34103ebdc3c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:45.915594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d83d2c-ab46-4c69-b208-34103ebdc3c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:45.917085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d83d2c-ab46-4c69-b208-34103ebdc3c4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:45.935276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 243/514] 10.222.0.64 () {56 vars in 1133 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:45.936138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d83d2c-ab46-4c69-b208-34103ebdc3c4 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:45.974811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 272/515] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:15:45 2022] POST /identity/v3/users => generated 247 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:15:45.975815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a0767-e15d-4bcb-afec-2e26b9d73def None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:46.101818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d83d2c-ab46-4c69-b208-34103ebdc3c4 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:46.127439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a0767-e15d-4bcb-afec-2e26b9d73def None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:46.154825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:46.168171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e861f2e-3530-4d63-bd30-cf34bb7770b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:46.169575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e861f2e-3530-4d63-bd30-cf34bb7770b9 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:46.171823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e861f2e-3530-4d63-bd30-cf34bb7770b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:46.171823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e861f2e-3530-4d63-bd30-cf34bb7770b9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:46.177770 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdeb243-497b-413c-8e04-97303a040d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:46.179267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdeb243-497b-413c-8e04-97303a040d47 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:46.179724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdeb243-497b-413c-8e04-97303a040d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:46.180198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdeb243-497b-413c-8e04-97303a040d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:46.185951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e861f2e-3530-4d63-bd30-cf34bb7770b9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:46.188832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e861f2e-3530-4d63-bd30-cf34bb7770b9 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:46.196044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:46.199519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdeb243-497b-413c-8e04-97303a040d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:46.201520 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdeb243-497b-413c-8e04-97303a040d47 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-9e861f2e-3530-4d63-bd30-cf34bb7770b9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:46.222254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:46.223650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 09:15:46.224723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:46.225222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 244/516] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:46.226363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e861f2e-3530-4d63-bd30-cf34bb7770b9 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-6bdeb243-497b-413c-8e04-97303a040d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:46.230172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:46.233576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:46.234510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:46.234510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:46.234510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:46.234510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:46.234510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 273/517] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:46.235052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdeb243-497b-413c-8e04-97303a040d47 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:46.369369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdeb243-497b-413c-8e04-97303a040d47 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:46.401188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e861f2e-3530-4d63-bd30-cf34bb7770b9 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:46.418480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e896e-a208-4b16-8b90-597c70fa0104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:46.419834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e896e-a208-4b16-8b90-597c70fa0104 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:46.420421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e896e-a208-4b16-8b90-597c70fa0104 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:46.420994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e896e-a208-4b16-8b90-597c70fa0104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:46.443044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665ef3e-f591-4f33-a4c7-85e905314552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:46.443850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665ef3e-f591-4f33-a4c7-85e905314552 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:46.444205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665ef3e-f591-4f33-a4c7-85e905314552 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:46.444552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665ef3e-f591-4f33-a4c7-85e905314552 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:46.445547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665ef3e-f591-4f33-a4c7-85e905314552 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:46.445931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665ef3e-f591-4f33-a4c7-85e905314552 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:46.447703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665ef3e-f591-4f33-a4c7-85e905314552 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:46.472703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 245/518] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:46.472703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665ef3e-f591-4f33-a4c7-85e905314552 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:46.526427 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:46.537644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e896e-a208-4b16-8b90-597c70fa0104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:46.539237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e896e-a208-4b16-8b90-597c70fa0104 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:46.555297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:46.628846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665ef3e-f591-4f33-a4c7-85e905314552 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:46.674500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7af81-3081-41e9-856f-78335bcc8297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:46.676120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7af81-3081-41e9-856f-78335bcc8297 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:46.676509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7af81-3081-41e9-856f-78335bcc8297 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:46.676892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7af81-3081-41e9-856f-78335bcc8297 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:46.677936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7af81-3081-41e9-856f-78335bcc8297 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:46.678471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7af81-3081-41e9-856f-78335bcc8297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:46.679963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7af81-3081-41e9-856f-78335bcc8297 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:46.689714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 274/519] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:46.690611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e896e-a208-4b16-8b90-597c70fa0104 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:46.709300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 246/520] 10.222.0.64 () {56 vars in 1139 bytes} [Mon Jun 27 09:15:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:46.709780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7af81-3081-41e9-856f-78335bcc8297 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:46.824658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:46.836645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e896e-a208-4b16-8b90-597c70fa0104 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:46.839430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7af81-3081-41e9-856f-78335bcc8297 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:46.848089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:46.876680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484cd84-e458-45b2-b98b-2fa91c72dffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:46.877704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484cd84-e458-45b2-b98b-2fa91c72dffe None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:46.878208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484cd84-e458-45b2-b98b-2fa91c72dffe None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:46.878661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484cd84-e458-45b2-b98b-2fa91c72dffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:46.879793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484cd84-e458-45b2-b98b-2fa91c72dffe None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=e33d1473077247cc97dcfd99b3da5599) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:46.880269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484cd84-e458-45b2-b98b-2fa91c72dffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:46.882121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484cd84-e458-45b2-b98b-2fa91c72dffe None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:46.899743 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 247/521] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:46 2022] GET /identity/v3/role_assignments?user.id=e33d1473077247cc97dcfd99b3da5599&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:46.901215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484cd84-e458-45b2-b98b-2fa91c72dffe None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:46.903562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce733c-01a8-4c59-8654-7ff2392a58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:46.904723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce733c-01a8-4c59-8654-7ff2392a58c9 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:46.905208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce733c-01a8-4c59-8654-7ff2392a58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:46.905691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce733c-01a8-4c59-8654-7ff2392a58c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:46.907047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce733c-01a8-4c59-8654-7ff2392a58c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:46.907627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce733c-01a8-4c59-8654-7ff2392a58c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:46.911007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce733c-01a8-4c59-8654-7ff2392a58c9 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:46.940114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 275/522] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:46.941769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce733c-01a8-4c59-8654-7ff2392a58c9 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:47.068444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484cd84-e458-45b2-b98b-2fa91c72dffe None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:47.110927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5b7fa-7283-4525-85fa-d36b4fb56412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:47.112083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5b7fa-7283-4525-85fa-d36b4fb56412 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:47.112551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5b7fa-7283-4525-85fa-d36b4fb56412 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:47.112974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5b7fa-7283-4525-85fa-d36b4fb56412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:47.119467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce733c-01a8-4c59-8654-7ff2392a58c9 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:47.126105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5b7fa-7283-4525-85fa-d36b4fb56412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:47.127732 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5b7fa-7283-4525-85fa-d36b4fb56412 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-c2a5b7fa-7283-4525-85fa-d36b4fb56412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:47.152953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:47.154574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:47.156339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:47.156339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:47.156339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:47.156339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:47.156339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 248/523] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:47.156339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5b7fa-7283-4525-85fa-d36b4fb56412 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:47.180496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4000e-eba2-42be-aea2-f9ac9cff5079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:47.182530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4000e-eba2-42be-aea2-f9ac9cff5079 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:47.183222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4000e-eba2-42be-aea2-f9ac9cff5079 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:47.183727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4000e-eba2-42be-aea2-f9ac9cff5079 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:47.194197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:47.218482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:47.282792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4000e-eba2-42be-aea2-f9ac9cff5079 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:47.287900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4000e-eba2-42be-aea2-f9ac9cff5079 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:47.305193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5b7fa-7283-4525-85fa-d36b4fb56412 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:47.369066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33df22a-9669-4376-b683-5ecc71fe33fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:47.370452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33df22a-9669-4376-b683-5ecc71fe33fa None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:47.370757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33df22a-9669-4376-b683-5ecc71fe33fa None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:47.371212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33df22a-9669-4376-b683-5ecc71fe33fa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:47.382964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33df22a-9669-4376-b683-5ecc71fe33fa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:47.387181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33df22a-9669-4376-b683-5ecc71fe33fa None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:47.407233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 276/524] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:47.408073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4000e-eba2-42be-aea2-f9ac9cff5079 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-f33df22a-9669-4376-b683-5ecc71fe33fa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:47.413390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:47.414669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:47.416024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 249/525] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:47.417098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33df22a-9669-4376-b683-5ecc71fe33fa None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:47.484143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:47.533360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4000e-eba2-42be-aea2-f9ac9cff5079 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:47.572884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:47.573319 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11fc15-e7e4-4caf-8b57-a3f95c886882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:47.574264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11fc15-e7e4-4caf-8b57-a3f95c886882 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:47.574756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11fc15-e7e4-4caf-8b57-a3f95c886882 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:47.575235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11fc15-e7e4-4caf-8b57-a3f95c886882 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:47.576273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11fc15-e7e4-4caf-8b57-a3f95c886882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:47.576797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11fc15-e7e4-4caf-8b57-a3f95c886882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:47.578443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11fc15-e7e4-4caf-8b57-a3f95c886882 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:47.591808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 277/526] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:47.593000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11fc15-e7e4-4caf-8b57-a3f95c886882 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:47.601100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33df22a-9669-4376-b683-5ecc71fe33fa None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:47.660755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad06cf-568b-40ed-8565-798dc63064e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:47.661775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad06cf-568b-40ed-8565-798dc63064e8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:47.662242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad06cf-568b-40ed-8565-798dc63064e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:47.662634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad06cf-568b-40ed-8565-798dc63064e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:47.677178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad06cf-568b-40ed-8565-798dc63064e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:47.679108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad06cf-568b-40ed-8565-798dc63064e8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-99ad06cf-568b-40ed-8565-798dc63064e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:47.724360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:47.725362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:47.726228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:47.726228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:47.726228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:47.726228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:47.731779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 250/527] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:47.731779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad06cf-568b-40ed-8565-798dc63064e8 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:47.733654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11fc15-e7e4-4caf-8b57-a3f95c886882 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:47.755832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:47.781134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a1498-7100-468d-b807-5d1d53b3c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:47.782285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a1498-7100-468d-b807-5d1d53b3c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:47.782783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a1498-7100-468d-b807-5d1d53b3c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:47.783281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a1498-7100-468d-b807-5d1d53b3c6c2 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:47.784469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a1498-7100-468d-b807-5d1d53b3c6c2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:47.785081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a1498-7100-468d-b807-5d1d53b3c6c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:47.786842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a1498-7100-468d-b807-5d1d53b3c6c2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:47.838150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 278/528] 10.222.0.64 () {56 vars in 1137 bytes} [Mon Jun 27 09:15:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:47.839489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a1498-7100-468d-b807-5d1d53b3c6c2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:47.937652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad06cf-568b-40ed-8565-798dc63064e8 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:47.981997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c27c6c-9fb8-42e0-9080-57b08556c9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:47.983394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c27c6c-9fb8-42e0-9080-57b08556c9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:47.984011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c27c6c-9fb8-42e0-9080-57b08556c9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:47.984516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c27c6c-9fb8-42e0-9080-57b08556c9f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:47.987512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:47.994717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c27c6c-9fb8-42e0-9080-57b08556c9f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:47.996772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c27c6c-9fb8-42e0-9080-57b08556c9f5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-a3c27c6c-9fb8-42e0-9080-57b08556c9f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:48.013859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:48.019729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 251/529] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:48.020634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c27c6c-9fb8-42e0-9080-57b08556c9f5 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:48.040292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a1498-7100-468d-b807-5d1d53b3c6c2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:48.104003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4675-d5ab-4888-bf83-9f3836b7e34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:48.105125 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4675-d5ab-4888-bf83-9f3836b7e34d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:48.105618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4675-d5ab-4888-bf83-9f3836b7e34d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:48.106141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4675-d5ab-4888-bf83-9f3836b7e34d None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:48.107105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f4675-d5ab-4888-bf83-9f3836b7e34d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:48.107662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f4675-d5ab-4888-bf83-9f3836b7e34d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:48.109377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f4675-d5ab-4888-bf83-9f3836b7e34d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:48.132017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 279/530] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:48.134193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4675-d5ab-4888-bf83-9f3836b7e34d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:48.172727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c27c6c-9fb8-42e0-9080-57b08556c9f5 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:48.197159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:48.218222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a25af4-fbd0-4302-83fe-a92f593a6375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:48.218885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a25af4-fbd0-4302-83fe-a92f593a6375 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:48.219170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a25af4-fbd0-4302-83fe-a92f593a6375 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:48.219448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a25af4-fbd0-4302-83fe-a92f593a6375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:48.292798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4675-d5ab-4888-bf83-9f3836b7e34d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:48.308373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:48.319001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a25af4-fbd0-4302-83fe-a92f593a6375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:48.320507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a25af4-fbd0-4302-83fe-a92f593a6375 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:48.338840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf714e0f-e29f-4127-b1a9-10bc875551dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:48.339564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf714e0f-e29f-4127-b1a9-10bc875551dd None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:48.339840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf714e0f-e29f-4127-b1a9-10bc875551dd None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:48.340123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf714e0f-e29f-4127-b1a9-10bc875551dd None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:48.340940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf714e0f-e29f-4127-b1a9-10bc875551dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:48.341253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf714e0f-e29f-4127-b1a9-10bc875551dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:48.342738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf714e0f-e29f-4127-b1a9-10bc875551dd None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:48.361815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 280/531] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:48.362997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf714e0f-e29f-4127-b1a9-10bc875551dd None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:48.441312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 252/532] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:48.442777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89c149-fb0b-4a5c-86b0-b1af16d458d0 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:48.443071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89c149-fb0b-4a5c-86b0-b1af16d458d0 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:48.443345 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89c149-fb0b-4a5c-86b0-b1af16d458d0 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:48.447747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 253/533] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:48.447747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89c149-fb0b-4a5c-86b0-b1af16d458d0 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:48.479966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:48.492542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf714e0f-e29f-4127-b1a9-10bc875551dd None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:48.535143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71c564-a060-4e0e-81c3-ee7ee095160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:48.535997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71c564-a060-4e0e-81c3-ee7ee095160c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:48.536268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71c564-a060-4e0e-81c3-ee7ee095160c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:48.536549 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71c564-a060-4e0e-81c3-ee7ee095160c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:48.544468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71c564-a060-4e0e-81c3-ee7ee095160c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:48.545948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71c564-a060-4e0e-81c3-ee7ee095160c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-dd71c564-a060-4e0e-81c3-ee7ee095160c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:48.562283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:48.571046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 281/534] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:48.572885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71c564-a060-4e0e-81c3-ee7ee095160c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:48.611153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:48.620156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89c149-fb0b-4a5c-86b0-b1af16d458d0 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:48.671392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68130397-638c-4ddf-92a3-9e1fc2766945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:48.672546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68130397-638c-4ddf-92a3-9e1fc2766945 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:48.673057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68130397-638c-4ddf-92a3-9e1fc2766945 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:48.673564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68130397-638c-4ddf-92a3-9e1fc2766945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:48.675266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68130397-638c-4ddf-92a3-9e1fc2766945 None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19e0fbe0b5814ad1ae95c65a64b632d6) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:48.676007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68130397-638c-4ddf-92a3-9e1fc2766945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:48.678803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68130397-638c-4ddf-92a3-9e1fc2766945 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:48.699656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 254/535] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:48 2022] GET /identity/v3/role_assignments?user.id=19e0fbe0b5814ad1ae95c65a64b632d6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:48.699840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71c564-a060-4e0e-81c3-ee7ee095160c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:48.699939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88679b-40f6-4563-ac69-6189da0bab30 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:48.699939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88679b-40f6-4563-ac69-6189da0bab30 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:48.699939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88679b-40f6-4563-ac69-6189da0bab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:48.723941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-dc88679b-40f6-4563-ac69-6189da0bab30 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:48.726145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-dc88679b-40f6-4563-ac69-6189da0bab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:48.746542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3059c-e9c0-44f3-b37e-29edaf357b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:48.748439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3059c-e9c0-44f3-b37e-29edaf357b68 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:48.749338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3059c-e9c0-44f3-b37e-29edaf357b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:48.749820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3059c-e9c0-44f3-b37e-29edaf357b68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:48.769683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3059c-e9c0-44f3-b37e-29edaf357b68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:48.771621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3059c-e9c0-44f3-b37e-29edaf357b68 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-12c3059c-e9c0-44f3-b37e-29edaf357b68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:48.797692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:48.798613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:48.799463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:48.799463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:48.799463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:48.799463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:48.807328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 282/536] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:48.807328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3059c-e9c0-44f3-b37e-29edaf357b68 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:48.850507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 255/537] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:48.851751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88679b-40f6-4563-ac69-6189da0bab30 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:48.859992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:48.980098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:48.996299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88679b-40f6-4563-ac69-6189da0bab30 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:49.024257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3059c-e9c0-44f3-b37e-29edaf357b68 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:49.051932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4d852-c884-4208-8182-4e04251f78b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:49.053672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4d852-c884-4208-8182-4e04251f78b5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.054459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4d852-c884-4208-8182-4e04251f78b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.054961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4d852-c884-4208-8182-4e04251f78b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.056235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4d852-c884-4208-8182-4e04251f78b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:49.057251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4d852-c884-4208-8182-4e04251f78b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:49.059402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4d852-c884-4208-8182-4e04251f78b5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:49.065033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad8788-a771-4af7-b86c-91b0c8d7ad3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:49.065742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad8788-a771-4af7-b86c-91b0c8d7ad3b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.066026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad8788-a771-4af7-b86c-91b0c8d7ad3b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.066304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad8788-a771-4af7-b86c-91b0c8d7ad3b None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.067110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad8788-a771-4af7-b86c-91b0c8d7ad3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:49.067422 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad8788-a771-4af7-b86c-91b0c8d7ad3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:49.068995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad8788-a771-4af7-b86c-91b0c8d7ad3b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:49.076028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 256/538] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:49.077977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4d852-c884-4208-8182-4e04251f78b5 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:49.083140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 283/539] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:49.084565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e8e66-8cfc-4eea-b9d8-7d676411051f None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.084858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e8e66-8cfc-4eea-b9d8-7d676411051f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.085135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e8e66-8cfc-4eea-b9d8-7d676411051f None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.086661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 284/540] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:49.087255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e8e66-8cfc-4eea-b9d8-7d676411051f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:49.164675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:49.212437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e8e66-8cfc-4eea-b9d8-7d676411051f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:49.243496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4d852-c884-4208-8182-4e04251f78b5 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:49.255675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bf64d-bd27-418a-b2f5-869cd3ae13f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:49.256487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bf64d-bd27-418a-b2f5-869cd3ae13f7 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.256830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bf64d-bd27-418a-b2f5-869cd3ae13f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.257121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bf64d-bd27-418a-b2f5-869cd3ae13f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.265761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bf64d-bd27-418a-b2f5-869cd3ae13f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:49.266335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bf64d-bd27-418a-b2f5-869cd3ae13f7 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-322bf64d-bd27-418a-b2f5-869cd3ae13f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:49.287306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:49.288246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:49.289077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:49.289077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:49.289077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:49.289077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:49.289077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:49.289077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:49.289077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:49.289077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:49.294533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 285/541] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:49.294919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bf64d-bd27-418a-b2f5-869cd3ae13f7 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:49.304084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d236fc3-5fb4-4618-bce5-99441240ec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:49.304926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d236fc3-5fb4-4618-bce5-99441240ec57 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.305235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d236fc3-5fb4-4618-bce5-99441240ec57 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.305515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d236fc3-5fb4-4618-bce5-99441240ec57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.319865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d236fc3-5fb4-4618-bce5-99441240ec57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-9d236fc3-5fb4-4618-bce5-99441240ec57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:49.328369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:49.339042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 257/542] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:49.342860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790c2c-d2d4-46e2-84bd-1de684cf967f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.346466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790c2c-d2d4-46e2-84bd-1de684cf967f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.346466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790c2c-d2d4-46e2-84bd-1de684cf967f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.356703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1b790c2c-d2d4-46e2-84bd-1de684cf967f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:49.357980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1b790c2c-d2d4-46e2-84bd-1de684cf967f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:49.361494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:49.438903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bf64d-bd27-418a-b2f5-869cd3ae13f7 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:49.439685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 258/543] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:49.441170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790c2c-d2d4-46e2-84bd-1de684cf967f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:49.474413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:49.491128 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e5398-af46-494a-9ac4-9d4aad80f790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:49.492137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e5398-af46-494a-9ac4-9d4aad80f790 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.492550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e5398-af46-494a-9ac4-9d4aad80f790 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.492946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e5398-af46-494a-9ac4-9d4aad80f790 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.494180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e5398-af46-494a-9ac4-9d4aad80f790 None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=2b87f03cae6c4ca98a82dd5d981ed224) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:49.494632 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e5398-af46-494a-9ac4-9d4aad80f790 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:49.496527 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e5398-af46-494a-9ac4-9d4aad80f790 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:49.512068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 286/544] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:49 2022] GET /identity/v3/role_assignments?user.id=2b87f03cae6c4ca98a82dd5d981ed224&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:49.514905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e5398-af46-494a-9ac4-9d4aad80f790 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:49.631847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790c2c-d2d4-46e2-84bd-1de684cf967f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:49.644690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e5398-af46-494a-9ac4-9d4aad80f790 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:49.685567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8e8d7-4f6a-4636-8132-96fbc8008448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:49.686263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8e8d7-4f6a-4636-8132-96fbc8008448 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.686550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8e8d7-4f6a-4636-8132-96fbc8008448 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.686828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8e8d7-4f6a-4636-8132-96fbc8008448 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.687816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8e8d7-4f6a-4636-8132-96fbc8008448 None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:49.687988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8e8d7-4f6a-4636-8132-96fbc8008448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:49.689374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8e8d7-4f6a-4636-8132-96fbc8008448 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:49.697081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02219b6a-5910-4e2a-8c00-4a1d12139ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:49.698209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02219b6a-5910-4e2a-8c00-4a1d12139ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.698689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02219b6a-5910-4e2a-8c00-4a1d12139ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.699400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02219b6a-5910-4e2a-8c00-4a1d12139ba7 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.700602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02219b6a-5910-4e2a-8c00-4a1d12139ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:49.701319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02219b6a-5910-4e2a-8c00-4a1d12139ba7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:49.703297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02219b6a-5910-4e2a-8c00-4a1d12139ba7 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:49.704217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 287/545] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:49.705236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8e8d7-4f6a-4636-8132-96fbc8008448 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:49.716471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:49.726757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 259/546] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:49.730211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02219b6a-5910-4e2a-8c00-4a1d12139ba7 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:49.794760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:49.889932 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8e8d7-4f6a-4636-8132-96fbc8008448 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:49.917140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02219b6a-5910-4e2a-8c00-4a1d12139ba7 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:49.962043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093f63a-3acb-4819-9772-f95972ace217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:49.962689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093f63a-3acb-4819-9772-f95972ace217 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.963194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093f63a-3acb-4819-9772-f95972ace217 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.963753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093f63a-3acb-4819-9772-f95972ace217 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:49.984954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093f63a-3acb-4819-9772-f95972ace217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:49.994701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9028-a6cc-4d1c-b72a-e7227fd79b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:49.996515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9028-a6cc-4d1c-b72a-e7227fd79b45 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:49.997252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9028-a6cc-4d1c-b72a-e7227fd79b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:49.998028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f9028-a6cc-4d1c-b72a-e7227fd79b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3093f63a-3acb-4819-9772-f95972ace217 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:50.000686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:50.002385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:50.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 288/547] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:50.006890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093f63a-3acb-4819-9772-f95972ace217 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:50.031078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9028-a6cc-4d1c-b72a-e7227fd79b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:50.034873 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f9028-a6cc-4d1c-b72a-e7227fd79b45 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-f07f9028-a6cc-4d1c-b72a-e7227fd79b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:50.082082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:50.083048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:50.092285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:50.092285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:50.092285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:50.092285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:50.092285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 260/548] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:50.092285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9028-a6cc-4d1c-b72a-e7227fd79b45 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:50.164958 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:50.182072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093f63a-3acb-4819-9772-f95972ace217 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:50.201566 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:50.220601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f9028-a6cc-4d1c-b72a-e7227fd79b45 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:50.267415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff34a50-bcca-4a97-9fac-dfc8f558083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:50.273614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff34a50-bcca-4a97-9fac-dfc8f558083a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:50.273970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff34a50-bcca-4a97-9fac-dfc8f558083a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:50.274258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff34a50-bcca-4a97-9fac-dfc8f558083a None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:50.275320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff34a50-bcca-4a97-9fac-dfc8f558083a None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:50.275659 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff34a50-bcca-4a97-9fac-dfc8f558083a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:50.276304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc8f96-0ed7-4244-a614-09c4ed9a56ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:50.277257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff34a50-bcca-4a97-9fac-dfc8f558083a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:50.279352 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc8f96-0ed7-4244-a614-09c4ed9a56ba None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:50.281023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc8f96-0ed7-4244-a614-09c4ed9a56ba None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:50.281813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc8f96-0ed7-4244-a614-09c4ed9a56ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:50.286149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc8f96-0ed7-4244-a614-09c4ed9a56ba None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=8c8abb981d1b4c82a43b5a3372f98398) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:50.287176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc8f96-0ed7-4244-a614-09c4ed9a56ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:50.292021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 261/549] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:50.293021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff34a50-bcca-4a97-9fac-dfc8f558083a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:50.296697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc8f96-0ed7-4244-a614-09c4ed9a56ba None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:50.319579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 289/550] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:50 2022] GET /identity/v3/role_assignments?user.id=8c8abb981d1b4c82a43b5a3372f98398&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:50.319579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc8f96-0ed7-4244-a614-09c4ed9a56ba None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:50.533567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff34a50-bcca-4a97-9fac-dfc8f558083a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:50.552754 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:50.555197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc8f96-0ed7-4244-a614-09c4ed9a56ba None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:50.586500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e663ce-fdfd-4fea-846b-947c82a154c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:50.587692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e663ce-fdfd-4fea-846b-947c82a154c0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:50.588217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e663ce-fdfd-4fea-846b-947c82a154c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:50.588706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e663ce-fdfd-4fea-846b-947c82a154c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:50.589664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e663ce-fdfd-4fea-846b-947c82a154c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:50.590233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e663ce-fdfd-4fea-846b-947c82a154c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:50.591980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e663ce-fdfd-4fea-846b-947c82a154c0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:50.608280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 262/551] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:50.614839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e663ce-fdfd-4fea-846b-947c82a154c0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:50.616948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa9d14-af17-47cd-86a2-7b323341a57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:50.617775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa9d14-af17-47cd-86a2-7b323341a57a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:50.618058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa9d14-af17-47cd-86a2-7b323341a57a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:50.618336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa9d14-af17-47cd-86a2-7b323341a57a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:50.642872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa9d14-af17-47cd-86a2-7b323341a57a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:50.644440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa9d14-af17-47cd-86a2-7b323341a57a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-a4fa9d14-af17-47cd-86a2-7b323341a57a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:50.688162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:50.689122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:50.690063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:50.690063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:50.690063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:50.690063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:50.690063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 290/552] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:50.695664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa9d14-af17-47cd-86a2-7b323341a57a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:50.740430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:50.744369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e663ce-fdfd-4fea-846b-947c82a154c0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:50.801156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2959cb0-db0d-4f7c-966d-72ea3ee4fcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:50.802536 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2959cb0-db0d-4f7c-966d-72ea3ee4fcdf None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:50.803066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2959cb0-db0d-4f7c-966d-72ea3ee4fcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:50.803572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2959cb0-db0d-4f7c-966d-72ea3ee4fcdf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:50.812321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2959cb0-db0d-4f7c-966d-72ea3ee4fcdf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:50.814705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2959cb0-db0d-4f7c-966d-72ea3ee4fcdf None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-a2959cb0-db0d-4f7c-966d-72ea3ee4fcdf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:50.842296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:50.843282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:15:50.844256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:50.852743 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 263/553] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:15:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:50.859043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2959cb0-db0d-4f7c-966d-72ea3ee4fcdf None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:50.888355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa9d14-af17-47cd-86a2-7b323341a57a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:50.910958 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:50.930631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e3b0f-9e7f-4a65-bcbd-4c27346be7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:50.931404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e3b0f-9e7f-4a65-bcbd-4c27346be7ad None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:50.931728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e3b0f-9e7f-4a65-bcbd-4c27346be7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:50.932030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e3b0f-9e7f-4a65-bcbd-4c27346be7ad None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:50.932806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e3b0f-9e7f-4a65-bcbd-4c27346be7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:50.933257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e3b0f-9e7f-4a65-bcbd-4c27346be7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:50.934725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e3b0f-9e7f-4a65-bcbd-4c27346be7ad None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:50.948472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 291/554] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:50.949991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35d98-2a56-49c4-886c-969b1a2a00ae None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:50.950328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35d98-2a56-49c4-886c-969b1a2a00ae None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:50.950587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35d98-2a56-49c4-886c-969b1a2a00ae None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:50.956430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 292/555] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:50.957408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35d98-2a56-49c4-886c-969b1a2a00ae None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:50.999079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2959cb0-db0d-4f7c-966d-72ea3ee4fcdf None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:51.039441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca832662-22ff-43de-97c6-af5bc0467fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:51.040526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca832662-22ff-43de-97c6-af5bc0467fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:51.041018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca832662-22ff-43de-97c6-af5bc0467fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:51.041497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca832662-22ff-43de-97c6-af5bc0467fa6 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:51.042594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca832662-22ff-43de-97c6-af5bc0467fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:51.043293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca832662-22ff-43de-97c6-af5bc0467fa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:51.045025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca832662-22ff-43de-97c6-af5bc0467fa6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:51.065197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 264/556] 10.222.0.64 () {56 vars in 1135 bytes} [Mon Jun 27 09:15:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:51.066857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555db49-b66b-4ba5-a5bb-3f8a07109186 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:51.067358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555db49-b66b-4ba5-a5bb-3f8a07109186 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:51.067899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555db49-b66b-4ba5-a5bb-3f8a07109186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:51.084495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:51.086936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7555db49-b66b-4ba5-a5bb-3f8a07109186 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:51.088472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7555db49-b66b-4ba5-a5bb-3f8a07109186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:51.103897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35d98-2a56-49c4-886c-969b1a2a00ae None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:51.153079 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafbe9f-5635-43f9-a90f-3b57556f01b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:51.156074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafbe9f-5635-43f9-a90f-3b57556f01b5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:51.157097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafbe9f-5635-43f9-a90f-3b57556f01b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:51.158277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafbe9f-5635-43f9-a90f-3b57556f01b5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:51.173347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafbe9f-5635-43f9-a90f-3b57556f01b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:51.175663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafbe9f-5635-43f9-a90f-3b57556f01b5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:51.186617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 265/557] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:51.187865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555db49-b66b-4ba5-a5bb-3f8a07109186 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-9cafbe9f-5635-43f9-a90f-3b57556f01b5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:51.200327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:51.201900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:51.202863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 293/558] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:51.207190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafbe9f-5635-43f9-a90f-3b57556f01b5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:51.218018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:51.329170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555db49-b66b-4ba5-a5bb-3f8a07109186 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:51.334466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafbe9f-5635-43f9-a90f-3b57556f01b5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:51.381091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27f765-f903-42ba-9e40-800521a8d36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:51.382799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27f765-f903-42ba-9e40-800521a8d36b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:51.383716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27f765-f903-42ba-9e40-800521a8d36b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:51.384553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27f765-f903-42ba-9e40-800521a8d36b None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:51.386541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27f765-f903-42ba-9e40-800521a8d36b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:51.387428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27f765-f903-42ba-9e40-800521a8d36b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:51.391004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27f765-f903-42ba-9e40-800521a8d36b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:51.395662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e3985-edda-4498-9b34-b71d7ea24456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:51.396834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e3985-edda-4498-9b34-b71d7ea24456 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:51.397328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e3985-edda-4498-9b34-b71d7ea24456 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:51.399298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e3985-edda-4498-9b34-b71d7ea24456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:51.425214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 294/559] 10.222.0.64 () {56 vars in 1137 bytes} [Mon Jun 27 09:15:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:51.426615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27f765-f903-42ba-9e40-800521a8d36b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:51.431331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:51.535093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e3985-edda-4498-9b34-b71d7ea24456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:51.537138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e3985-edda-4498-9b34-b71d7ea24456 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:51.564052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:51.617328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27f765-f903-42ba-9e40-800521a8d36b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:51.668036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03caf4a-526b-4048-8a05-d727242873d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:51.670002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03caf4a-526b-4048-8a05-d727242873d4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:51.670911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03caf4a-526b-4048-8a05-d727242873d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:51.671468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03caf4a-526b-4048-8a05-d727242873d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:51.685917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03caf4a-526b-4048-8a05-d727242873d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-e03caf4a-526b-4048-8a05-d727242873d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:51.696925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:51.697902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:51.698762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:51.702498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 295/560] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:51.703637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03caf4a-526b-4048-8a05-d727242873d4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:51.719666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 266/561] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:51.720668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e3985-edda-4498-9b34-b71d7ea24456 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:51.800274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:51.844429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e3985-edda-4498-9b34-b71d7ea24456 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:51.863591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03caf4a-526b-4048-8a05-d727242873d4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:51.867128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:51.882600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07515c77-e50f-4ad8-b639-a7e4ae3e0d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:51.883125 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07515c77-e50f-4ad8-b639-a7e4ae3e0d35 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:51.883406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07515c77-e50f-4ad8-b639-a7e4ae3e0d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:51.883751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07515c77-e50f-4ad8-b639-a7e4ae3e0d35 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:51.884513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07515c77-e50f-4ad8-b639-a7e4ae3e0d35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:51.884815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07515c77-e50f-4ad8-b639-a7e4ae3e0d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:51.886203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07515c77-e50f-4ad8-b639-a7e4ae3e0d35 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:51.899575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 267/562] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:51.899575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07515c77-e50f-4ad8-b639-a7e4ae3e0d35 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:51.909214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c121c-5e94-4560-abdb-c255026ef1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:51.910386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c121c-5e94-4560-abdb-c255026ef1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:51.910674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c121c-5e94-4560-abdb-c255026ef1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:51.910953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c121c-5e94-4560-abdb-c255026ef1c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:51.922334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c121c-5e94-4560-abdb-c255026ef1c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:51.923850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c121c-5e94-4560-abdb-c255026ef1c6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-c27c121c-5e94-4560-abdb-c255026ef1c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:51.944221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:51.945229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:51.946368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:51.946368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:51.946368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:51.946368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:51.946368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 296/563] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:51.946748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc567a-4775-4107-8e60-e285ce020c03 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:51.947774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc567a-4775-4107-8e60-e285ce020c03 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:51.947774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc567a-4775-4107-8e60-e285ce020c03 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:51.948773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 297/564] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:51.953634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc567a-4775-4107-8e60-e285ce020c03 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:52.035617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07515c77-e50f-4ad8-b639-a7e4ae3e0d35 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:52.074522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f77e4d-654f-490e-9d21-cc8cb9b47173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:52.075340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f77e4d-654f-490e-9d21-cc8cb9b47173 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.075668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f77e4d-654f-490e-9d21-cc8cb9b47173 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.075946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f77e4d-654f-490e-9d21-cc8cb9b47173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.083127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc567a-4775-4107-8e60-e285ce020c03 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:52.087921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f77e4d-654f-490e-9d21-cc8cb9b47173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:52.089392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f77e4d-654f-490e-9d21-cc8cb9b47173 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-f6f77e4d-654f-490e-9d21-cc8cb9b47173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:52.107661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:52.109113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:52.110390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:52.110390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:52.110390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:52.110390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:52.110390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 268/565] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:52.110390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f77e4d-654f-490e-9d21-cc8cb9b47173 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:52.111230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:52.144123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb0cbc-0e09-4e22-8c3b-969a1cd4408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:52.144366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb0cbc-0e09-4e22-8c3b-969a1cd4408f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.144563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb0cbc-0e09-4e22-8c3b-969a1cd4408f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.144862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb0cbc-0e09-4e22-8c3b-969a1cd4408f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.159828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb0cbc-0e09-4e22-8c3b-969a1cd4408f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:52.161342 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb0cbc-0e09-4e22-8c3b-969a1cd4408f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:52.165246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-90fb0cbc-0e09-4e22-8c3b-969a1cd4408f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:52.181616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:52.183025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:52.186012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:52.186012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:52.186012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:52.186012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:52.187766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 298/566] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:52.189313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d0a2d-44e4-4e3e-94c0-dae00ab8dc8b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.189603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d0a2d-44e4-4e3e-94c0-dae00ab8dc8b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.189879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d0a2d-44e4-4e3e-94c0-dae00ab8dc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.209265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2c9d0a2d-44e4-4e3e-94c0-dae00ab8dc8b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:52.210509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2c9d0a2d-44e4-4e3e-94c0-dae00ab8dc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:52.303763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f77e4d-654f-490e-9d21-cc8cb9b47173 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:52.305611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 299/567] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:52.311850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d0a2d-44e4-4e3e-94c0-dae00ab8dc8b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:52.380001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa103769-7f4b-4b44-89e4-ac1bb2537c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:52.380799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa103769-7f4b-4b44-89e4-ac1bb2537c47 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.381081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa103769-7f4b-4b44-89e4-ac1bb2537c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.381370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa103769-7f4b-4b44-89e4-ac1bb2537c47 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.382094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa103769-7f4b-4b44-89e4-ac1bb2537c47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:52.382400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa103769-7f4b-4b44-89e4-ac1bb2537c47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:52.383828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa103769-7f4b-4b44-89e4-ac1bb2537c47 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:52.401034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 269/568] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:52.401034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa103769-7f4b-4b44-89e4-ac1bb2537c47 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:52.454944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:52.467696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d0a2d-44e4-4e3e-94c0-dae00ab8dc8b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:52.524499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac60abf-7ea5-46ff-805a-3747899503f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:52.525852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac60abf-7ea5-46ff-805a-3747899503f7 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.526317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac60abf-7ea5-46ff-805a-3747899503f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.526779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac60abf-7ea5-46ff-805a-3747899503f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.528147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac60abf-7ea5-46ff-805a-3747899503f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:52.528660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac60abf-7ea5-46ff-805a-3747899503f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:52.531031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac60abf-7ea5-46ff-805a-3747899503f7 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:52.551860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 300/569] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:52.553765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac60abf-7ea5-46ff-805a-3747899503f7 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:52.562669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:52.574714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa103769-7f4b-4b44-89e4-ac1bb2537c47 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:52.644609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b1a49-d95d-4d21-8915-ce4c1aad101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:52.645799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b1a49-d95d-4d21-8915-ce4c1aad101d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.646478 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b1a49-d95d-4d21-8915-ce4c1aad101d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.646808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b1a49-d95d-4d21-8915-ce4c1aad101d None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.648241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b1a49-d95d-4d21-8915-ce4c1aad101d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:52.648788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b1a49-d95d-4d21-8915-ce4c1aad101d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:52.651884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b1a49-d95d-4d21-8915-ce4c1aad101d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:52.675483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 270/570] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:52.676869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b1a49-d95d-4d21-8915-ce4c1aad101d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:52.704530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac60abf-7ea5-46ff-805a-3747899503f7 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:52.742323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab52f7-850a-415b-b1fe-8625fce43d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:52.743143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab52f7-850a-415b-b1fe-8625fce43d05 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.743472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab52f7-850a-415b-b1fe-8625fce43d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.743813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab52f7-850a-415b-b1fe-8625fce43d05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.756512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab52f7-850a-415b-b1fe-8625fce43d05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-d0ab52f7-850a-415b-b1fe-8625fce43d05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:52.764686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:52.766569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 301/571] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:52.769371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab52f7-850a-415b-b1fe-8625fce43d05 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:52.792010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:52.849864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b1a49-d95d-4d21-8915-ce4c1aad101d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:52.893769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab52f7-850a-415b-b1fe-8625fce43d05 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:52.895593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f9cb-afb6-465b-99b0-9f2666144316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:52.897048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f9cb-afb6-465b-99b0-9f2666144316 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.897525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f9cb-afb6-465b-99b0-9f2666144316 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.897985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f9cb-afb6-465b-99b0-9f2666144316 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.907021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290f9cb-afb6-465b-99b0-9f2666144316 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:52.908466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290f9cb-afb6-465b-99b0-9f2666144316 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-1290f9cb-afb6-465b-99b0-9f2666144316 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:52.923942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:52.925949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:52.926891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:52.926891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:52.926891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:52.926891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:52.926891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:52.926891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:52.926891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:52.926891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:52.926891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 271/572] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:52.927653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ef4ec-9061-4686-80f9-c3afdbc595f5 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.927764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ef4ec-9061-4686-80f9-c3afdbc595f5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.928126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ef4ec-9061-4686-80f9-c3afdbc595f5 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.929346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 272/573] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:52.930093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ef4ec-9061-4686-80f9-c3afdbc595f5 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:52.942653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca626b-8198-4aaa-846a-a8beb9016e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:52.943467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca626b-8198-4aaa-846a-a8beb9016e10 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:52.943792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca626b-8198-4aaa-846a-a8beb9016e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:52.944069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca626b-8198-4aaa-846a-a8beb9016e10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:52.951604 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca626b-8198-4aaa-846a-a8beb9016e10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:52.953092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca626b-8198-4aaa-846a-a8beb9016e10 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:52.954529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-efca626b-8198-4aaa-846a-a8beb9016e10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:52.968846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:52.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 302/574] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:52.971426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca626b-8198-4aaa-846a-a8beb9016e10 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:53.057532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:53.062672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ef4ec-9061-4686-80f9-c3afdbc595f5 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:53.082501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca626b-8198-4aaa-846a-a8beb9016e10 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:53.102382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74aa04-d683-4857-9d7d-54702242b7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:53.103184 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74aa04-d683-4857-9d7d-54702242b7dc None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:53.103463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74aa04-d683-4857-9d7d-54702242b7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:53.103776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74aa04-d683-4857-9d7d-54702242b7dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:53.111978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74aa04-d683-4857-9d7d-54702242b7dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:53.113466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74aa04-d683-4857-9d7d-54702242b7dc None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-2c74aa04-d683-4857-9d7d-54702242b7dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:53.129446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:53.131379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 273/575] 10.222.0.64 () {56 vars in 1093 bytes} [Mon Jun 27 09:15:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:53.133267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74aa04-d683-4857-9d7d-54702242b7dc None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:53.137331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043f966-1858-4676-933d-a8b46a5d743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:53.138323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043f966-1858-4676-933d-a8b46a5d743e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:53.138785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043f966-1858-4676-933d-a8b46a5d743e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:53.139245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043f966-1858-4676-933d-a8b46a5d743e None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:53.140124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043f966-1858-4676-933d-a8b46a5d743e None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:53.140480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043f966-1858-4676-933d-a8b46a5d743e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:53.142132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043f966-1858-4676-933d-a8b46a5d743e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:53.152275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 303/576] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:53.153658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6740d1-385b-4de2-946a-b9b09e305254 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:53.153952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6740d1-385b-4de2-946a-b9b09e305254 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:53.154231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6740d1-385b-4de2-946a-b9b09e305254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:53.172693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2c6740d1-385b-4de2-946a-b9b09e305254 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:53.173883 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2c6740d1-385b-4de2-946a-b9b09e305254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:53.243718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:53.264245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 304/577] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:53.265128 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6740d1-385b-4de2-946a-b9b09e305254 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:53.288182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74aa04-d683-4857-9d7d-54702242b7dc None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:53.334724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7a50-13b1-4cd0-90cc-717878ba859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:53.335711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7a50-13b1-4cd0-90cc-717878ba859c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:53.336131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7a50-13b1-4cd0-90cc-717878ba859c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:53.336587 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7a50-13b1-4cd0-90cc-717878ba859c None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:53.338014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d7a50-13b1-4cd0-90cc-717878ba859c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:53.338381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d7a50-13b1-4cd0-90cc-717878ba859c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:53.341165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d7a50-13b1-4cd0-90cc-717878ba859c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:53.357384 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 274/578] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:53.358537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7a50-13b1-4cd0-90cc-717878ba859c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:53.367325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:53.417850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6740d1-385b-4de2-946a-b9b09e305254 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:53.465121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10d00-3ced-4503-ad7f-59c428ee8b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:53.466009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10d00-3ced-4503-ad7f-59c428ee8b6e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:53.466333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10d00-3ced-4503-ad7f-59c428ee8b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:53.466757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10d00-3ced-4503-ad7f-59c428ee8b6e None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:53.467611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10d00-3ced-4503-ad7f-59c428ee8b6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:53.467933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10d00-3ced-4503-ad7f-59c428ee8b6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:53.469442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10d00-3ced-4503-ad7f-59c428ee8b6e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:53.482397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 305/579] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:53.483256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10d00-3ced-4503-ad7f-59c428ee8b6e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:53.494573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7a50-13b1-4cd0-90cc-717878ba859c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:53.534596 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450a6b4-ee95-48b4-8914-d1bce2154512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:53.535292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450a6b4-ee95-48b4-8914-d1bce2154512 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:53.535604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450a6b4-ee95-48b4-8914-d1bce2154512 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:53.535892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450a6b4-ee95-48b4-8914-d1bce2154512 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:53.536763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450a6b4-ee95-48b4-8914-d1bce2154512 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:53.537078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450a6b4-ee95-48b4-8914-d1bce2154512 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:53.538598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450a6b4-ee95-48b4-8914-d1bce2154512 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:53.540639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:53.559291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 275/580] 10.222.0.64 () {56 vars in 1133 bytes} [Mon Jun 27 09:15:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:53.560144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450a6b4-ee95-48b4-8914-d1bce2154512 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:53.605233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10d00-3ced-4503-ad7f-59c428ee8b6e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:53.636415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:53.645070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef7e66-6a7c-4468-b1df-b565dbe34565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:53.647853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7e66-6a7c-4468-b1df-b565dbe34565 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:53.647853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7e66-6a7c-4468-b1df-b565dbe34565 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:53.647853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef7e66-6a7c-4468-b1df-b565dbe34565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:53.658430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef7e66-6a7c-4468-b1df-b565dbe34565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:53.660140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef7e66-6a7c-4468-b1df-b565dbe34565 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-adef7e66-6a7c-4468-b1df-b565dbe34565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:53.688659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:53.691229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:53.693356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:53.693356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:53.693356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:53.693356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:53.694133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 306/581] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:53.695470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef7e66-6a7c-4468-b1df-b565dbe34565 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:53.715290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450a6b4-ee95-48b4-8914-d1bce2154512 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:53.772499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c777e-cc9e-437d-aec0-959e9a770e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:53.773856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c777e-cc9e-437d-aec0-959e9a770e8c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:53.774325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c777e-cc9e-437d-aec0-959e9a770e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:53.774818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c777e-cc9e-437d-aec0-959e9a770e8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:53.790495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c777e-cc9e-437d-aec0-959e9a770e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-074c777e-cc9e-437d-aec0-959e9a770e8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:53.798685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:53.801123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 276/582] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:53.802716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c777e-cc9e-437d-aec0-959e9a770e8c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:53.820308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef7e66-6a7c-4468-b1df-b565dbe34565 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:53.867008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aca6ac-5b4a-4d99-8e03-98eba51fab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:53.868269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aca6ac-5b4a-4d99-8e03-98eba51fab2e None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:53.868662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aca6ac-5b4a-4d99-8e03-98eba51fab2e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:53.869064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aca6ac-5b4a-4d99-8e03-98eba51fab2e None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/e33d1473077247cc97dcfd99b3da5599/roles/ee8bde816f274202886522710bb8fdf2` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:53.875164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:53.890138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aca6ac-5b4a-4d99-8e03-98eba51fab2e None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=e33d1473077247cc97dcfd99b3da5599, role_id=ee8bde816f274202886522710bb8fdf2)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:53.892119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aca6ac-5b4a-4d99-8e03-98eba51fab2e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:53.952256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c777e-cc9e-437d-aec0-959e9a770e8c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:54.016377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b55903-37cf-48b3-9ce8-1f69797c4952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:54.017542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b55903-37cf-48b3-9ce8-1f69797c4952 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:54.018113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b55903-37cf-48b3-9ce8-1f69797c4952 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:54.018594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b55903-37cf-48b3-9ce8-1f69797c4952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:54.020440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b55903-37cf-48b3-9ce8-1f69797c4952 None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=70b499a32516486d9a2bed34f7e66b76) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:54.021023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b55903-37cf-48b3-9ce8-1f69797c4952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:54.023860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b55903-37cf-48b3-9ce8-1f69797c4952 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:54.035887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 277/583] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:15:53 2022] GET /identity/v3/role_assignments?user.id=70b499a32516486d9a2bed34f7e66b76&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:54.037428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b55903-37cf-48b3-9ce8-1f69797c4952 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:54.055078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 307/584] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:15:53 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/e33d1473077247cc97dcfd99b3da5599/roles/ee8bde816f274202886522710bb8fdf2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:54.056374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aca6ac-5b4a-4d99-8e03-98eba51fab2e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:54.124247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:54.233246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aca6ac-5b4a-4d99-8e03-98eba51fab2e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:54.263901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b55903-37cf-48b3-9ce8-1f69797c4952 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:54.283372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0ff2e-6806-4be9-8f42-03b563c924a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:54.284657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0ff2e-6806-4be9-8f42-03b563c924a1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:54.284942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0ff2e-6806-4be9-8f42-03b563c924a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:54.285221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0ff2e-6806-4be9-8f42-03b563c924a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:54.288896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0ff2e-6806-4be9-8f42-03b563c924a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:54.288896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0ff2e-6806-4be9-8f42-03b563c924a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:54.288896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0ff2e-6806-4be9-8f42-03b563c924a1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:54.300495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:54.305754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 308/585] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:54.307811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0ff2e-6806-4be9-8f42-03b563c924a1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:54.317355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893ba9-f49f-4d0b-ba8d-45400293145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:54.318288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893ba9-f49f-4d0b-ba8d-45400293145e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:54.318617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893ba9-f49f-4d0b-ba8d-45400293145e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:54.318897 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28893ba9-f49f-4d0b-ba8d-45400293145e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:54.333205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28893ba9-f49f-4d0b-ba8d-45400293145e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:54.335377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28893ba9-f49f-4d0b-ba8d-45400293145e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-28893ba9-f49f-4d0b-ba8d-45400293145e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:54.365352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:54.366980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:54.368244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:54.368244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:54.368244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:54.368244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:54.371287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 278/586] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:54.378734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893ba9-f49f-4d0b-ba8d-45400293145e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:54.490109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:54.505402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0ff2e-6806-4be9-8f42-03b563c924a1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:54.511406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28893ba9-f49f-4d0b-ba8d-45400293145e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:54.579405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8400e0-27a8-4123-8a3a-cf94fdd383de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:54.580790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8400e0-27a8-4123-8a3a-cf94fdd383de None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:54.581657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8400e0-27a8-4123-8a3a-cf94fdd383de None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:54.582165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8400e0-27a8-4123-8a3a-cf94fdd383de None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:54.584854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b42c5-b04c-4933-a415-c43a99b34e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:54.586296 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b42c5-b04c-4933-a415-c43a99b34e03 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:54.587073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b42c5-b04c-4933-a415-c43a99b34e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:54.588036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b42c5-b04c-4933-a415-c43a99b34e03 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:54.589769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b42c5-b04c-4933-a415-c43a99b34e03 None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:54.590586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b42c5-b04c-4933-a415-c43a99b34e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:54.593644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b42c5-b04c-4933-a415-c43a99b34e03 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:54.595310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8400e0-27a8-4123-8a3a-cf94fdd383de None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:54.597699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8400e0-27a8-4123-8a3a-cf94fdd383de None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:54.610359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 309/587] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:54.612556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b42c5-b04c-4933-a415-c43a99b34e03 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-af8400e0-27a8-4123-8a3a-cf94fdd383de None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:54.614970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:54.616790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 09:15:54.618657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:54.620877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 279/588] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:54.621817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8400e0-27a8-4123-8a3a-cf94fdd383de None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:54.662870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:54.771367 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b42c5-b04c-4933-a415-c43a99b34e03 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:54.776313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8400e0-27a8-4123-8a3a-cf94fdd383de None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:54.810945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f1787-e699-42f7-89bd-75bb86081a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:54.811795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f1787-e699-42f7-89bd-75bb86081a80 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:54.812155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f1787-e699-42f7-89bd-75bb86081a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:54.812506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f1787-e699-42f7-89bd-75bb86081a80 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:54.813320 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f1787-e699-42f7-89bd-75bb86081a80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:54.813704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f1787-e699-42f7-89bd-75bb86081a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:54.815209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f1787-e699-42f7-89bd-75bb86081a80 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:54.828978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08580e-d72e-4b4f-896d-1a0de47fc315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:54.834690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 310/589] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:54.834690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f1787-e699-42f7-89bd-75bb86081a80 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:54.835710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08580e-d72e-4b4f-896d-1a0de47fc315 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:54.836223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08580e-d72e-4b4f-896d-1a0de47fc315 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:54.836691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08580e-d72e-4b4f-896d-1a0de47fc315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:54.852468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:54.858550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08580e-d72e-4b4f-896d-1a0de47fc315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:54.860384 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08580e-d72e-4b4f-896d-1a0de47fc315 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-ab08580e-d72e-4b4f-896d-1a0de47fc315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:54.886742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:54.888462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:54.889594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:54.889594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:54.889594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:54.889594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:54.892142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 280/590] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:54.894616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08580e-d72e-4b4f-896d-1a0de47fc315 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:54.963302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f1787-e699-42f7-89bd-75bb86081a80 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:55.002857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f159d7-dc02-439b-af6b-112c1dc76d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:55.003641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f159d7-dc02-439b-af6b-112c1dc76d7e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:55.003926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f159d7-dc02-439b-af6b-112c1dc76d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:55.004197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f159d7-dc02-439b-af6b-112c1dc76d7e None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:55.005112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f159d7-dc02-439b-af6b-112c1dc76d7e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:55.005420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f159d7-dc02-439b-af6b-112c1dc76d7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:55.007003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f159d7-dc02-439b-af6b-112c1dc76d7e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:55.019314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:55.027196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 311/591] 10.222.0.64 () {56 vars in 1139 bytes} [Mon Jun 27 09:15:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:55.028116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f159d7-dc02-439b-af6b-112c1dc76d7e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:55.043807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08580e-d72e-4b4f-896d-1a0de47fc315 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:55.086128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1e1c-1c50-482d-869e-83333d6ce789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:55.087379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1e1c-1c50-482d-869e-83333d6ce789 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:55.087984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1e1c-1c50-482d-869e-83333d6ce789 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:55.088477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1e1c-1c50-482d-869e-83333d6ce789 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:55.097825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1e1c-1c50-482d-869e-83333d6ce789 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:55.099913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1e1c-1c50-482d-869e-83333d6ce789 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:55.106736 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-179b1e1c-1c50-482d-869e-83333d6ce789 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:55.117647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:55.118610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:55.119982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:55.119982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:55.119982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:55.119982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:55.119982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:55.119982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:55.119982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:55.119982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:55.122313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 281/592] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:55.123431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1e1c-1c50-482d-869e-83333d6ce789 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:55.141585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f159d7-dc02-439b-af6b-112c1dc76d7e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:55.181758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf44920-76a4-48e3-8e8c-9c4c957e74f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:55.182584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf44920-76a4-48e3-8e8c-9c4c957e74f3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:55.186329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf44920-76a4-48e3-8e8c-9c4c957e74f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:55.186329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf44920-76a4-48e3-8e8c-9c4c957e74f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:55.186329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf44920-76a4-48e3-8e8c-9c4c957e74f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:55.186329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf44920-76a4-48e3-8e8c-9c4c957e74f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:55.186329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf44920-76a4-48e3-8e8c-9c4c957e74f3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:55.203855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 312/593] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:55.205018 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf44920-76a4-48e3-8e8c-9c4c957e74f3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:55.249058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1e1c-1c50-482d-869e-83333d6ce789 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:55.301051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac0bc9-48d2-4709-8461-773dc0cc7d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:55.302356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac0bc9-48d2-4709-8461-773dc0cc7d44 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:55.302848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac0bc9-48d2-4709-8461-773dc0cc7d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:55.303430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac0bc9-48d2-4709-8461-773dc0cc7d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:55.316877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac0bc9-48d2-4709-8461-773dc0cc7d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:55.318847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac0bc9-48d2-4709-8461-773dc0cc7d44 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:55.320120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:55.320770 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf44920-76a4-48e3-8e8c-9c4c957e74f3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-afac0bc9-48d2-4709-8461-773dc0cc7d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:55.346441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:55.348032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:55.348945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:55.348945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:55.348945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:55.348945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:55.348945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 282/594] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:55.348945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac0bc9-48d2-4709-8461-773dc0cc7d44 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:55.354384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:55.360760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13638fb7-9f8b-4b55-a689-bca09cca9062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:55.361620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13638fb7-9f8b-4b55-a689-bca09cca9062 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:55.362335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13638fb7-9f8b-4b55-a689-bca09cca9062 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:55.362335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13638fb7-9f8b-4b55-a689-bca09cca9062 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:55.363193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13638fb7-9f8b-4b55-a689-bca09cca9062 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:55.363646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13638fb7-9f8b-4b55-a689-bca09cca9062 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:55.365204 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13638fb7-9f8b-4b55-a689-bca09cca9062 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:55.378945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 313/595] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:55.380899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13638fb7-9f8b-4b55-a689-bca09cca9062 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:55.498260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac0bc9-48d2-4709-8461-773dc0cc7d44 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:55.505002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13638fb7-9f8b-4b55-a689-bca09cca9062 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:55.551966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec0e17-eebf-40a4-a6c3-7c2c2c07b3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:55.554527 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec0e17-eebf-40a4-a6c3-7c2c2c07b3ec None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:55.556073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec0e17-eebf-40a4-a6c3-7c2c2c07b3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:55.557202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec0e17-eebf-40a4-a6c3-7c2c2c07b3ec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:55.569171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf0444-f27e-451f-8bbd-1b861be796aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:55.570022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf0444-f27e-451f-8bbd-1b861be796aa None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:55.570432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf0444-f27e-451f-8bbd-1b861be796aa None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:55.570813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf0444-f27e-451f-8bbd-1b861be796aa None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:55.571764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf0444-f27e-451f-8bbd-1b861be796aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:55.572177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf0444-f27e-451f-8bbd-1b861be796aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:55.573867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf0444-f27e-451f-8bbd-1b861be796aa None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:55.577970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec0e17-eebf-40a4-a6c3-7c2c2c07b3ec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:55.580586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec0e17-eebf-40a4-a6c3-7c2c2c07b3ec None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:55.594841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 314/596] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:55.594841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf0444-f27e-451f-8bbd-1b861be796aa None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0aec0e17-eebf-40a4-a6c3-7c2c2c07b3ec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:55.619334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:55.621489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 09:15:55.624344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:55.634308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 283/597] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:55.634308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec0e17-eebf-40a4-a6c3-7c2c2c07b3ec None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:55.681332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:55.702844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:55.740870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf0444-f27e-451f-8bbd-1b861be796aa None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:55.775934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec0e17-eebf-40a4-a6c3-7c2c2c07b3ec None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:55.783485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5327e-3fef-4dc8-9dd2-748932b65b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:55.784949 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5327e-3fef-4dc8-9dd2-748932b65b85 None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:55.785231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5327e-3fef-4dc8-9dd2-748932b65b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:55.785509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5327e-3fef-4dc8-9dd2-748932b65b85 None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19e0fbe0b5814ad1ae95c65a64b632d6/roles/ee8bde816f274202886522710bb8fdf2` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:55.804674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5327e-3fef-4dc8-9dd2-748932b65b85 None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=19e0fbe0b5814ad1ae95c65a64b632d6, role_id=ee8bde816f274202886522710bb8fdf2)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:55.806072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5327e-3fef-4dc8-9dd2-748932b65b85 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:55.820285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb5941-6f40-4cc2-b342-dbae6787f00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:55.821187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb5941-6f40-4cc2-b342-dbae6787f00f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:55.821477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb5941-6f40-4cc2-b342-dbae6787f00f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:55.822581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb5941-6f40-4cc2-b342-dbae6787f00f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:55.830230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb5941-6f40-4cc2-b342-dbae6787f00f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:55.832404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb5941-6f40-4cc2-b342-dbae6787f00f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-78cb5941-6f40-4cc2-b342-dbae6787f00f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:55.849462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:55.856684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 284/598] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:55.857769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb5941-6f40-4cc2-b342-dbae6787f00f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:55.895698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 315/599] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:15:55 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19e0fbe0b5814ad1ae95c65a64b632d6/roles/ee8bde816f274202886522710bb8fdf2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:55.896915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5327e-3fef-4dc8-9dd2-748932b65b85 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:56.055174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb5941-6f40-4cc2-b342-dbae6787f00f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:56.091213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:56.136206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:56.141575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5327e-3fef-4dc8-9dd2-748932b65b85 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:56.144136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14eb9d-819b-4842-80f4-a34785ee6e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:56.144862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14eb9d-819b-4842-80f4-a34785ee6e1e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:56.145153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14eb9d-819b-4842-80f4-a34785ee6e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:56.145427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14eb9d-819b-4842-80f4-a34785ee6e1e None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:56.147235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14eb9d-819b-4842-80f4-a34785ee6e1e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:56.147235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14eb9d-819b-4842-80f4-a34785ee6e1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:56.148170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14eb9d-819b-4842-80f4-a34785ee6e1e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:56.177088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 285/600] 10.222.0.64 () {56 vars in 1137 bytes} [Mon Jun 27 09:15:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:56.178764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14eb9d-819b-4842-80f4-a34785ee6e1e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:56.193312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3fb1d-c529-4116-a954-8802bcfcac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:56.194933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3fb1d-c529-4116-a954-8802bcfcac4d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:56.195463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3fb1d-c529-4116-a954-8802bcfcac4d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:56.195961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3fb1d-c529-4116-a954-8802bcfcac4d None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:56.197171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3fb1d-c529-4116-a954-8802bcfcac4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:56.198129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3fb1d-c529-4116-a954-8802bcfcac4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:56.200225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3fb1d-c529-4116-a954-8802bcfcac4d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:56.223413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 316/601] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:56.231112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3fb1d-c529-4116-a954-8802bcfcac4d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:56.413010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14eb9d-819b-4842-80f4-a34785ee6e1e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:56.481151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76c040-1b70-4bd0-b45e-439fcf06d7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:56.482464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76c040-1b70-4bd0-b45e-439fcf06d7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:56.482975 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76c040-1b70-4bd0-b45e-439fcf06d7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:56.484290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76c040-1b70-4bd0-b45e-439fcf06d7a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:56.499447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76c040-1b70-4bd0-b45e-439fcf06d7a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:56.501535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76c040-1b70-4bd0-b45e-439fcf06d7a5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-4b76c040-1b70-4bd0-b45e-439fcf06d7a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:56.523385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:56.524619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:56.525830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:56.525830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:56.525830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:56.525830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:56.525830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 286/602] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:56.526641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d9c09-3210-431d-a1f9-cc0ec2b09789 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:56.526931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d9c09-3210-431d-a1f9-cc0ec2b09789 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:56.527205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d9c09-3210-431d-a1f9-cc0ec2b09789 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:56.528853 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 287/603] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:56.534217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179ab21-c00d-4945-9386-d781ea168460 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:56.534540 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179ab21-c00d-4945-9386-d781ea168460 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:56.534767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179ab21-c00d-4945-9386-d781ea168460 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:56.536206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 288/604] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:56.540354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8c4cd-2996-4e7d-8568-c9822885a05b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:56.540354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8c4cd-2996-4e7d-8568-c9822885a05b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:56.540354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8c4cd-2996-4e7d-8568-c9822885a05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:56.559803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3fb1d-c529-4116-a954-8802bcfcac4d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:56.569090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e0b8c4cd-2996-4e7d-8568-c9822885a05b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:56.570945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e0b8c4cd-2996-4e7d-8568-c9822885a05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:56.598951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:56.615489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ea275-f316-4052-9017-728d83444486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:56.617189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea275-f316-4052-9017-728d83444486 None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:56.617563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea275-f316-4052-9017-728d83444486 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:56.618015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ea275-f316-4052-9017-728d83444486 None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/2b87f03cae6c4ca98a82dd5d981ed224/roles/ee8bde816f274202886522710bb8fdf2` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:56.645720 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ea275-f316-4052-9017-728d83444486 None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=2b87f03cae6c4ca98a82dd5d981ed224, role_id=ee8bde816f274202886522710bb8fdf2)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:56.647960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ea275-f316-4052-9017-728d83444486 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:56.676930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 289/605] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:56.677801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8c4cd-2996-4e7d-8568-c9822885a05b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:56.756486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 317/606] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:15:56 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/2b87f03cae6c4ca98a82dd5d981ed224/roles/ee8bde816f274202886522710bb8fdf2 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:56.757475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3b241-6ffb-45a7-8ca9-86c5841ae3e7 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:56.757819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3b241-6ffb-45a7-8ca9-86c5841ae3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:56.758099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3b241-6ffb-45a7-8ca9-86c5841ae3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:56.797859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3fb3b241-6ffb-45a7-8ca9-86c5841ae3e7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:56.799114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3fb3b241-6ffb-45a7-8ca9-86c5841ae3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:56.806614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:56.847251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8c4cd-2996-4e7d-8568-c9822885a05b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:56.896290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 318/607] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:56.896599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ecf4b-f60d-4b41-9f59-d0bb7aaa0596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:56.897583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ecf4b-f60d-4b41-9f59-d0bb7aaa0596 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:56.897942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ecf4b-f60d-4b41-9f59-d0bb7aaa0596 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:56.898308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ecf4b-f60d-4b41-9f59-d0bb7aaa0596 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:56.898558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3b241-6ffb-45a7-8ca9-86c5841ae3e7 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:56.899204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ecf4b-f60d-4b41-9f59-d0bb7aaa0596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:56.899610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ecf4b-f60d-4b41-9f59-d0bb7aaa0596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:56.901669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ecf4b-f60d-4b41-9f59-d0bb7aaa0596 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:56.923913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 290/608] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:56.926553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ecf4b-f60d-4b41-9f59-d0bb7aaa0596 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:56.957811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:57.066528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3b241-6ffb-45a7-8ca9-86c5841ae3e7 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:57.080069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ecf4b-f60d-4b41-9f59-d0bb7aaa0596 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:57.121005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aada327-10ab-4e8f-91db-7d055aad3047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:57.121836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aada327-10ab-4e8f-91db-7d055aad3047 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:57.122132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aada327-10ab-4e8f-91db-7d055aad3047 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:57.122383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aada327-10ab-4e8f-91db-7d055aad3047 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:57.133585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:57.134646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7276831-7c19-4a4a-8b78-cd5dc4a2d191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:57.136413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7276831-7c19-4a4a-8b78-cd5dc4a2d191 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:57.137097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7276831-7c19-4a4a-8b78-cd5dc4a2d191 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:57.137789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aada327-10ab-4e8f-91db-7d055aad3047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:57.138186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7276831-7c19-4a4a-8b78-cd5dc4a2d191 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-6aada327-10ab-4e8f-91db-7d055aad3047 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:57.147560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:57.152142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 319/609] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:57.153387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aada327-10ab-4e8f-91db-7d055aad3047 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:57.157716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7276831-7c19-4a4a-8b78-cd5dc4a2d191 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:57.159639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7276831-7c19-4a4a-8b78-cd5dc4a2d191 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-d7276831-7c19-4a4a-8b78-cd5dc4a2d191 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:57.177316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:57.178348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:57.179269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:57.179269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:57.179269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:57.179269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:57.179269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:57.179269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:57.179269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:57.179269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:57.185972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 291/610] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:57.185972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7276831-7c19-4a4a-8b78-cd5dc4a2d191 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:57.293392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aada327-10ab-4e8f-91db-7d055aad3047 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:57.309618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:57.343026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb56b3c-65ec-40e4-a854-1203a64acf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:57.344213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7276831-7c19-4a4a-8b78-cd5dc4a2d191 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:57.344570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56b3c-65ec-40e4-a854-1203a64acf1e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:57.345087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56b3c-65ec-40e4-a854-1203a64acf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:57.345557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56b3c-65ec-40e4-a854-1203a64acf1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:57.361161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb56b3c-65ec-40e4-a854-1203a64acf1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-4bb56b3c-65ec-40e4-a854-1203a64acf1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:57.373041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:57.374039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:57.374895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:57.382462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 320/611] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:57.382462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb56b3c-65ec-40e4-a854-1203a64acf1e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:57.397488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7e74e-e5e0-4774-b937-c5ce2437bb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:57.398178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7e74e-e5e0-4774-b937-c5ce2437bb20 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:57.398465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7e74e-e5e0-4774-b937-c5ce2437bb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:57.398747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7e74e-e5e0-4774-b937-c5ce2437bb20 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:57.399640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7e74e-e5e0-4774-b937-c5ce2437bb20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:57.400476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7e74e-e5e0-4774-b937-c5ce2437bb20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:57.401732 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7e74e-e5e0-4774-b937-c5ce2437bb20 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:57.411614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 292/612] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:57.412928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7e74e-e5e0-4774-b937-c5ce2437bb20 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:57.456706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:57.501840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb56b3c-65ec-40e4-a854-1203a64acf1e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:57.540778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429383d-6605-455b-a46a-7b73e0381ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:57.541377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429383d-6605-455b-a46a-7b73e0381ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:57.541662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429383d-6605-455b-a46a-7b73e0381ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:57.541941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429383d-6605-455b-a46a-7b73e0381ff5 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:57.542757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429383d-6605-455b-a46a-7b73e0381ff5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:57.543065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429383d-6605-455b-a46a-7b73e0381ff5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:57.544574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429383d-6605-455b-a46a-7b73e0381ff5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:57.547566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7e74e-e5e0-4774-b937-c5ce2437bb20 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:57.563810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 321/613] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:15:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:57.565000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429383d-6605-455b-a46a-7b73e0381ff5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:57.588598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266c7f3-2d16-4db7-b55f-2094f2016017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:57.589823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266c7f3-2d16-4db7-b55f-2094f2016017 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:57.590334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266c7f3-2d16-4db7-b55f-2094f2016017 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:57.590838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266c7f3-2d16-4db7-b55f-2094f2016017 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:57.591891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266c7f3-2d16-4db7-b55f-2094f2016017 None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:57.592550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266c7f3-2d16-4db7-b55f-2094f2016017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:57.594450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266c7f3-2d16-4db7-b55f-2094f2016017 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:57.596619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:57.606740 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 293/614] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:57.613257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266c7f3-2d16-4db7-b55f-2094f2016017 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:57.683917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429383d-6605-455b-a46a-7b73e0381ff5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:57.706577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:57.728468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f7212-d174-4ebe-918f-874d9397ff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:57.729876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f7212-d174-4ebe-918f-874d9397ff01 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:57.730383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f7212-d174-4ebe-918f-874d9397ff01 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:57.730901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f7212-d174-4ebe-918f-874d9397ff01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:57.733164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266c7f3-2d16-4db7-b55f-2094f2016017 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:57.751690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f7212-d174-4ebe-918f-874d9397ff01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:57.754833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f7212-d174-4ebe-918f-874d9397ff01 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:57.773131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0075f8-4843-438d-94ed-06ed088c7ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:57.774394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0075f8-4843-438d-94ed-06ed088c7ded None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:57.774865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0075f8-4843-438d-94ed-06ed088c7ded None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:57.775630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0075f8-4843-438d-94ed-06ed088c7ded None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/8c8abb981d1b4c82a43b5a3372f98398/roles/ee8bde816f274202886522710bb8fdf2` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-829f7212-d174-4ebe-918f-874d9397ff01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:57.792452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:57.793373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:57.794285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:57.794285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:57.794285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:57.794285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:57.796127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 322/615] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:57.797132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0075f8-4843-438d-94ed-06ed088c7ded None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=8c8abb981d1b4c82a43b5a3372f98398, role_id=ee8bde816f274202886522710bb8fdf2)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:57.797776 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f7212-d174-4ebe-918f-874d9397ff01 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:57.799064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0075f8-4843-438d-94ed-06ed088c7ded None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:57.917855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f7212-d174-4ebe-918f-874d9397ff01 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:57.948514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 294/616] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:15:57 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/8c8abb981d1b4c82a43b5a3372f98398/roles/ee8bde816f274202886522710bb8fdf2 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:15:57.950425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0075f8-4843-438d-94ed-06ed088c7ded None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:57.981375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3746f-f578-4949-8416-50ce9f172e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:57.982218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3746f-f578-4949-8416-50ce9f172e16 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:57.982669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3746f-f578-4949-8416-50ce9f172e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:57.982952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3746f-f578-4949-8416-50ce9f172e16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.009360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3746f-f578-4949-8416-50ce9f172e16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:58.010850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3746f-f578-4949-8416-50ce9f172e16 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-09e3746f-f578-4949-8416-50ce9f172e16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:58.049086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:58.050616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:58.051698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:58.051698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:58.051698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:58.051698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:58.051698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 323/617] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:58.058923 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:58.059779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3746f-f578-4949-8416-50ce9f172e16 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:58.111965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:58.163167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0075f8-4843-438d-94ed-06ed088c7ded None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:58.224989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412ebdd-4bae-4772-a881-3267ae573dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:58.226640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412ebdd-4bae-4772-a881-3267ae573dff None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.227145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412ebdd-4bae-4772-a881-3267ae573dff None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.227712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412ebdd-4bae-4772-a881-3267ae573dff None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.231137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412ebdd-4bae-4772-a881-3267ae573dff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:58.231292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412ebdd-4bae-4772-a881-3267ae573dff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:58.234953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412ebdd-4bae-4772-a881-3267ae573dff None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:58.245672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3746f-f578-4949-8416-50ce9f172e16 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:58.249936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 295/618] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:58.251172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf488eb-98b8-4d26-b53d-7221b8dd89be None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.251475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf488eb-98b8-4d26-b53d-7221b8dd89be None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.251777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf488eb-98b8-4d26-b53d-7221b8dd89be None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.253430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 296/619] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:58.260479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd6ef3-2296-4d14-a02e-23b9ccd92fdc None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.262265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd6ef3-2296-4d14-a02e-23b9ccd92fdc None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.262545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd6ef3-2296-4d14-a02e-23b9ccd92fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.285043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d7bd6ef3-2296-4d14-a02e-23b9ccd92fdc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:58.286510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d7bd6ef3-2296-4d14-a02e-23b9ccd92fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:58.300829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646b5a-90f7-4ab2-a6fd-8f1cb31f29a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:58.301967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646b5a-90f7-4ab2-a6fd-8f1cb31f29a1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.302861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646b5a-90f7-4ab2-a6fd-8f1cb31f29a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.303654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646b5a-90f7-4ab2-a6fd-8f1cb31f29a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.304706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60646b5a-90f7-4ab2-a6fd-8f1cb31f29a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:58.305355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60646b5a-90f7-4ab2-a6fd-8f1cb31f29a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:58.307279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60646b5a-90f7-4ab2-a6fd-8f1cb31f29a1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:58.323633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 324/620] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:15:58.325000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646b5a-90f7-4ab2-a6fd-8f1cb31f29a1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:58.375906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 297/621] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:58.376998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd6ef3-2296-4d14-a02e-23b9ccd92fdc None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:58.425004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:58.444947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:58.462600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646b5a-90f7-4ab2-a6fd-8f1cb31f29a1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:58.500398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14429a61-7522-4538-8bc5-637a1c78fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:58.502055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14429a61-7522-4538-8bc5-637a1c78fab4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.503240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14429a61-7522-4538-8bc5-637a1c78fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.503756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14429a61-7522-4538-8bc5-637a1c78fab4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.504813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd6ef3-2296-4d14-a02e-23b9ccd92fdc None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:58.514336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14429a61-7522-4538-8bc5-637a1c78fab4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:58.517095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14429a61-7522-4538-8bc5-637a1c78fab4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-14429a61-7522-4538-8bc5-637a1c78fab4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:58.538241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:58.539303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:15:58.540328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:58.546411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 325/622] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:15:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:58.546411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14429a61-7522-4538-8bc5-637a1c78fab4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:58.550386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a436a-8a21-4a67-ae63-989509f9cf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:58.551565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a436a-8a21-4a67-ae63-989509f9cf1e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.552078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a436a-8a21-4a67-ae63-989509f9cf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.552576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a436a-8a21-4a67-ae63-989509f9cf1e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.562929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a436a-8a21-4a67-ae63-989509f9cf1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:58.565889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a436a-8a21-4a67-ae63-989509f9cf1e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-737a436a-8a21-4a67-ae63-989509f9cf1e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:15:58.586668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:58.587704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:58.588656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 298/623] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:15:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:58.589237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a436a-8a21-4a67-ae63-989509f9cf1e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:58.674529 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14429a61-7522-4538-8bc5-637a1c78fab4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:58.696009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:58.703089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a436a-8a21-4a67-ae63-989509f9cf1e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:58.724470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a05fe-18b2-4b7e-bb8b-7e79de21f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:58.725865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a05fe-18b2-4b7e-bb8b-7e79de21f991 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.726356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a05fe-18b2-4b7e-bb8b-7e79de21f991 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.727117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a05fe-18b2-4b7e-bb8b-7e79de21f991 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.741954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a05fe-18b2-4b7e-bb8b-7e79de21f991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:58.744919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e412e-63e0-4fef-ae46-4524776e5285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:58.745673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e412e-63e0-4fef-ae46-4524776e5285 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.745961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e412e-63e0-4fef-ae46-4524776e5285 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.746241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e412e-63e0-4fef-ae46-4524776e5285 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.747140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e412e-63e0-4fef-ae46-4524776e5285 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:58.747456 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e412e-63e0-4fef-ae46-4524776e5285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:58.748938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e412e-63e0-4fef-ae46-4524776e5285 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-c98a05fe-18b2-4b7e-bb8b-7e79de21f991 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:58.753771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:58.754740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:58.755628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:58.763380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 326/624] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:58.763380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a05fe-18b2-4b7e-bb8b-7e79de21f991 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:58.769332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:58.781408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 299/625] 10.222.0.64 () {56 vars in 1135 bytes} [Mon Jun 27 09:15:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:58.781408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e412e-63e0-4fef-ae46-4524776e5285 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:58.910568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a05fe-18b2-4b7e-bb8b-7e79de21f991 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:58.921975 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e412e-63e0-4fef-ae46-4524776e5285 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:58.960582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224a4f9-8950-4668-ac2e-14e6669be30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:58.961457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1049823-babc-4724-9494-44847ca7a7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:58.961889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224a4f9-8950-4668-ac2e-14e6669be30d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.962145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049823-babc-4724-9494-44847ca7a7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.962530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049823-babc-4724-9494-44847ca7a7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.962820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049823-babc-4724-9494-44847ca7a7a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.962949 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224a4f9-8950-4668-ac2e-14e6669be30d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.963160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224a4f9-8950-4668-ac2e-14e6669be30d None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:58.963680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1049823-babc-4724-9494-44847ca7a7a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:58.963997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1049823-babc-4724-9494-44847ca7a7a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:58.964095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224a4f9-8950-4668-ac2e-14e6669be30d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:58.964448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224a4f9-8950-4668-ac2e-14e6669be30d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:58.965424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1049823-babc-4724-9494-44847ca7a7a6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:58.965893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224a4f9-8950-4668-ac2e-14e6669be30d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:58.975000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 300/626] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:58.976212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1049823-babc-4724-9494-44847ca7a7a6 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:58.992986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 327/627] 10.222.0.64 () {56 vars in 1137 bytes} [Mon Jun 27 09:15:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:58.993921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bfbf4-7827-4702-8f20-9159be0f1ab4 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:58.994192 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bfbf4-7827-4702-8f20-9159be0f1ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:58.994473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bfbf4-7827-4702-8f20-9159be0f1ab4 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.002185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 328/628] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:15:59.002185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bfbf4-7827-4702-8f20-9159be0f1ab4 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:59.012388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:59.076460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:59.117965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1049823-babc-4724-9494-44847ca7a7a6 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:59.130602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bfbf4-7827-4702-8f20-9159be0f1ab4 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:59.157671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93dfe3-e1c1-469c-a19c-dceb488d50c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:59.158526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93dfe3-e1c1-469c-a19c-dceb488d50c0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:59.158818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93dfe3-e1c1-469c-a19c-dceb488d50c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:59.159638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93dfe3-e1c1-469c-a19c-dceb488d50c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.169834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e4f6d-e66c-46d5-8075-de0b4c52db39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:59.170893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e4f6d-e66c-46d5-8075-de0b4c52db39 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:59.171209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e4f6d-e66c-46d5-8075-de0b4c52db39 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:59.171434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93dfe3-e1c1-469c-a19c-dceb488d50c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:59.171887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e4f6d-e66c-46d5-8075-de0b4c52db39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.172943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93dfe3-e1c1-469c-a19c-dceb488d50c0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:59.189882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e4f6d-e66c-46d5-8075-de0b4c52db39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:59.192220 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e4f6d-e66c-46d5-8075-de0b4c52db39 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-3f93dfe3-e1c1-469c-a19c-dceb488d50c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:59.199692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:59.201037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:59.202220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:59.202220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:59.202220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:59.202220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:59.203843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 301/629] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:59.204609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93dfe3-e1c1-469c-a19c-dceb488d50c0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-799e4f6d-e66c-46d5-8075-de0b4c52db39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:15:59.217362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:59.218350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:59.219341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:15:59.219341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:15:59.219341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:15:59.219341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:59.219341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 329/630] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:59.220591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa521a35-12b7-4880-8a8e-4621f2d7358e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:59.220591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa521a35-12b7-4880-8a8e-4621f2d7358e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:59.220892 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa521a35-12b7-4880-8a8e-4621f2d7358e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.239439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-fa521a35-12b7-4880-8a8e-4621f2d7358e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:15:59.243425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fa521a35-12b7-4880-8a8e-4621f2d7358e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:15:59.301348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:59.327764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 330/631] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:15:59.327764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa521a35-12b7-4880-8a8e-4621f2d7358e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:59.338118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93dfe3-e1c1-469c-a19c-dceb488d50c0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:59.378730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cddef6-4a29-4731-88b8-e60b1a3986eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:59.380260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cddef6-4a29-4731-88b8-e60b1a3986eb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:59.380778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cddef6-4a29-4731-88b8-e60b1a3986eb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:59.381251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cddef6-4a29-4731-88b8-e60b1a3986eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.385899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:59.451880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa521a35-12b7-4880-8a8e-4621f2d7358e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:59.456462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cddef6-4a29-4731-88b8-e60b1a3986eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:59.457957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cddef6-4a29-4731-88b8-e60b1a3986eb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:59.494700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4443cc-053a-4783-bf37-b8773fecb71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:59.495429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4443cc-053a-4783-bf37-b8773fecb71e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:59.495791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4443cc-053a-4783-bf37-b8773fecb71e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:59.496073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4443cc-053a-4783-bf37-b8773fecb71e None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.496800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4443cc-053a-4783-bf37-b8773fecb71e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:59.497138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4443cc-053a-4783-bf37-b8773fecb71e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:59.498730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4443cc-053a-4783-bf37-b8773fecb71e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:59.511668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 331/632] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:59.516850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4443cc-053a-4783-bf37-b8773fecb71e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:59.561450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:59.601183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 302/633] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:15:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:15:59.604543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cddef6-4a29-4731-88b8-e60b1a3986eb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:59.638824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4443cc-053a-4783-bf37-b8773fecb71e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:59.641335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:59.676766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53ef04-a8a7-4df1-88d2-eb492cfaae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:59.677424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53ef04-a8a7-4df1-88d2-eb492cfaae24 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:59.677737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53ef04-a8a7-4df1-88d2-eb492cfaae24 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:59.677973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53ef04-a8a7-4df1-88d2-eb492cfaae24 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.678695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53ef04-a8a7-4df1-88d2-eb492cfaae24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:15:59.678992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53ef04-a8a7-4df1-88d2-eb492cfaae24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:59.680555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53ef04-a8a7-4df1-88d2-eb492cfaae24 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:59.694413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 332/634] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:15:59.695283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53ef04-a8a7-4df1-88d2-eb492cfaae24 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:59.752790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cddef6-4a29-4731-88b8-e60b1a3986eb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:59.796007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771e536-8abe-435a-8919-ba19b02870c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:59.797118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771e536-8abe-435a-8919-ba19b02870c4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:59.797701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771e536-8abe-435a-8919-ba19b02870c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:59.797982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771e536-8abe-435a-8919-ba19b02870c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.808966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771e536-8abe-435a-8919-ba19b02870c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0771e536-8abe-435a-8919-ba19b02870c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:15:59.817885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:59.818825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:15:59.820282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:15:59.820935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53ef04-a8a7-4df1-88d2-eb492cfaae24 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:59.821010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 303/635] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:15:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:59.821010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771e536-8abe-435a-8919-ba19b02870c4 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:59.851934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:59.860848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d337aa7-3ce3-401a-9475-c1d6de5a8e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:59.866042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337aa7-3ce3-401a-9475-c1d6de5a8e8c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:59.866042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337aa7-3ce3-401a-9475-c1d6de5a8e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:59.866042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337aa7-3ce3-401a-9475-c1d6de5a8e8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.869763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d337aa7-3ce3-401a-9475-c1d6de5a8e8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:59.871574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d337aa7-3ce3-401a-9475-c1d6de5a8e8c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-1d337aa7-3ce3-401a-9475-c1d6de5a8e8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:15:59.885563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:15:59.886480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:15:59.887372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:15:59.887372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:15:59.887372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:15:59.887372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:15:59.887372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:15:59.887372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:15:59.887372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:15:59.887372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:15:59.887372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 333/636] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:15:59.888473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d337aa7-3ce3-401a-9475-c1d6de5a8e8c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:15:59.918943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:15:59.931462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771e536-8abe-435a-8919-ba19b02870c4 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:15:59.972950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96e1a2-78f4-46fe-926a-51715bf9b85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:15:59.973742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96e1a2-78f4-46fe-926a-51715bf9b85b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:15:59.974022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96e1a2-78f4-46fe-926a-51715bf9b85b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:15:59.974294 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96e1a2-78f4-46fe-926a-51715bf9b85b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:15:59.984849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96e1a2-78f4-46fe-926a-51715bf9b85b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:15:59.986309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96e1a2-78f4-46fe-926a-51715bf9b85b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-1f96e1a2-78f4-46fe-926a-51715bf9b85b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:00.005217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:00.006133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:00.007117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:00.007117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:00.007117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:00.007117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:00.007117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 304/637] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:00.007390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96e1a2-78f4-46fe-926a-51715bf9b85b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:00.015485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d337aa7-3ce3-401a-9475-c1d6de5a8e8c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:00.068336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d95cbc-56c8-45f5-9b47-05fcb1a09189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:00.069151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d95cbc-56c8-45f5-9b47-05fcb1a09189 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.069457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d95cbc-56c8-45f5-9b47-05fcb1a09189 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.069734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d95cbc-56c8-45f5-9b47-05fcb1a09189 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.079004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d95cbc-56c8-45f5-9b47-05fcb1a09189 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:00.080824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d95cbc-56c8-45f5-9b47-05fcb1a09189 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-d2d95cbc-56c8-45f5-9b47-05fcb1a09189 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:00.096118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:00.097348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:00.098344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 334/638] 10.222.0.64 () {56 vars in 1093 bytes} [Mon Jun 27 09:15:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:00.098954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d95cbc-56c8-45f5-9b47-05fcb1a09189 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:00.118909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:00.145414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96e1a2-78f4-46fe-926a-51715bf9b85b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:00.185627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ee138-aacf-42bf-ba83-fe7e57b29126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:00.186789 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ee138-aacf-42bf-ba83-fe7e57b29126 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.187492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ee138-aacf-42bf-ba83-fe7e57b29126 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.187492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ee138-aacf-42bf-ba83-fe7e57b29126 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.193322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ee138-aacf-42bf-ba83-fe7e57b29126 None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:00.193322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ee138-aacf-42bf-ba83-fe7e57b29126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:00.193322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ee138-aacf-42bf-ba83-fe7e57b29126 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:00.207128 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:00.214154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 305/639] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:16:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:00.215042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ee138-aacf-42bf-ba83-fe7e57b29126 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:00.225689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d95cbc-56c8-45f5-9b47-05fcb1a09189 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:00.270575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad123d-d789-4c52-85ae-b2eb6f37eebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:00.271151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad123d-d789-4c52-85ae-b2eb6f37eebd None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.271436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad123d-d789-4c52-85ae-b2eb6f37eebd None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.271752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad123d-d789-4c52-85ae-b2eb6f37eebd None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.272567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad123d-d789-4c52-85ae-b2eb6f37eebd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:00.272881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad123d-d789-4c52-85ae-b2eb6f37eebd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:00.274350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad123d-d789-4c52-85ae-b2eb6f37eebd None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:00.289138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 335/640] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:00.289846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad123d-d789-4c52-85ae-b2eb6f37eebd None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:00.381734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ee138-aacf-42bf-ba83-fe7e57b29126 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:00.397781 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad123d-d789-4c52-85ae-b2eb6f37eebd None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:00.417818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdaaf1d-4533-4045-bb0a-19ad4c98931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:00.418488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdaaf1d-4533-4045-bb0a-19ad4c98931d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.418780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdaaf1d-4533-4045-bb0a-19ad4c98931d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.419053 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdaaf1d-4533-4045-bb0a-19ad4c98931d None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.419816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdaaf1d-4533-4045-bb0a-19ad4c98931d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:00.420126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdaaf1d-4533-4045-bb0a-19ad4c98931d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:00.421528 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdaaf1d-4533-4045-bb0a-19ad4c98931d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:00.434187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 306/641] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:00.434633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5709e6-4ec9-40ad-bc02-15cb86655517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:00.435315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5709e6-4ec9-40ad-bc02-15cb86655517 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.435601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62827e5-53d6-4aca-9edf-020321aec84b None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.435692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5709e6-4ec9-40ad-bc02-15cb86655517 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.435891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5709e6-4ec9-40ad-bc02-15cb86655517 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.436026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62827e5-53d6-4aca-9edf-020321aec84b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.436347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62827e5-53d6-4aca-9edf-020321aec84b None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.436797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5709e6-4ec9-40ad-bc02-15cb86655517 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:00.437108 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5709e6-4ec9-40ad-bc02-15cb86655517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:00.439569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 307/642] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:00.439569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62827e5-53d6-4aca-9edf-020321aec84b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:00.439798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5709e6-4ec9-40ad-bc02-15cb86655517 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:00.443294 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:00.459092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 336/643] 10.222.0.64 () {56 vars in 1133 bytes} [Mon Jun 27 09:16:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:00.460190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5709e6-4ec9-40ad-bc02-15cb86655517 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:00.463100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:00.546389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62827e5-53d6-4aca-9edf-020321aec84b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:00.561001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5709e6-4ec9-40ad-bc02-15cb86655517 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:00.582680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025c4fa-8dcf-458d-aa2f-a3a3f4d4430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:00.583471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025c4fa-8dcf-458d-aa2f-a3a3f4d4430e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.583776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025c4fa-8dcf-458d-aa2f-a3a3f4d4430e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.584047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025c4fa-8dcf-458d-aa2f-a3a3f4d4430e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.594882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025c4fa-8dcf-458d-aa2f-a3a3f4d4430e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:00.596470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025c4fa-8dcf-458d-aa2f-a3a3f4d4430e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:00.597590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c459830-b502-4790-bab1-17fa5cef37d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:00.598492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c459830-b502-4790-bab1-17fa5cef37d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.598773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c459830-b502-4790-bab1-17fa5cef37d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.599058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c459830-b502-4790-bab1-17fa5cef37d2 None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/70b499a32516486d9a2bed34f7e66b76/roles/ee8bde816f274202886522710bb8fdf2` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-6025c4fa-8dcf-458d-aa2f-a3a3f4d4430e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:00.615425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:00.616402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:00.618123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:00.618123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:00.618123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:00.618123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:00.618355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c459830-b502-4790-bab1-17fa5cef37d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=70b499a32516486d9a2bed34f7e66b76, role_id=ee8bde816f274202886522710bb8fdf2)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:00.618428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 308/644] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:00.618494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c459830-b502-4790-bab1-17fa5cef37d2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:00.618560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025c4fa-8dcf-458d-aa2f-a3a3f4d4430e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:00.677530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 337/645] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:16:00 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/70b499a32516486d9a2bed34f7e66b76/roles/ee8bde816f274202886522710bb8fdf2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:16:00.678972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee267b-7760-4135-8f78-187965e11961 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.679258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee267b-7760-4135-8f78-187965e11961 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.680230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee267b-7760-4135-8f78-187965e11961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.689226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:00.714824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d6ee267b-7760-4135-8f78-187965e11961 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:00.716115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d6ee267b-7760-4135-8f78-187965e11961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:00.744829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025c4fa-8dcf-458d-aa2f-a3a3f4d4430e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:00.791784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bad902-e644-4128-876c-a0166ef8ea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:00.792641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bad902-e644-4128-876c-a0166ef8ea17 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:00.792934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bad902-e644-4128-876c-a0166ef8ea17 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:00.793221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bad902-e644-4128-876c-a0166ef8ea17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:00.796894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:00.806127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bad902-e644-4128-876c-a0166ef8ea17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:00.809302 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bad902-e644-4128-876c-a0166ef8ea17 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:00.836114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 338/646] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-c3bad902-e644-4128-876c-a0166ef8ea17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:00.837535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:00.838883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:00.841337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:00.841337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:00.841337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:00.841337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:00.841337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:00.841337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:00.841337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:00.841337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:00.841337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 309/647] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:00.842120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bad902-e644-4128-876c-a0166ef8ea17 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:00.843332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee267b-7760-4135-8f78-187965e11961 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:01.022794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee267b-7760-4135-8f78-187965e11961 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:01.023640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bad902-e644-4128-876c-a0166ef8ea17 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:01.042207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:01.079039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9a5ab-fca0-4c63-9f76-9c65beb42425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:01.081321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9a5ab-fca0-4c63-9f76-9c65beb42425 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:01.082270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9a5ab-fca0-4c63-9f76-9c65beb42425 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:01.082840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef48c1-f88c-48aa-a51f-1e7143fe3de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:01.083388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9a5ab-fca0-4c63-9f76-9c65beb42425 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:01.083577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef48c1-f88c-48aa-a51f-1e7143fe3de9 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:01.083838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef48c1-f88c-48aa-a51f-1e7143fe3de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:01.084107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef48c1-f88c-48aa-a51f-1e7143fe3de9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:01.085734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9a5ab-fca0-4c63-9f76-9c65beb42425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:01.086635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9a5ab-fca0-4c63-9f76-9c65beb42425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:01.089895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9a5ab-fca0-4c63-9f76-9c65beb42425 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:01.116088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 310/648] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:01.118226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9a5ab-fca0-4c63-9f76-9c65beb42425 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:01.161352 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:01.189426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef48c1-f88c-48aa-a51f-1e7143fe3de9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:01.191275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef48c1-f88c-48aa-a51f-1e7143fe3de9 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:01.296359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9a5ab-fca0-4c63-9f76-9c65beb42425 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:01.343690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cad33c-05d4-40b8-a9b0-05707bdd2320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:01.344313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cad33c-05d4-40b8-a9b0-05707bdd2320 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:01.344608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cad33c-05d4-40b8-a9b0-05707bdd2320 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:01.344947 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cad33c-05d4-40b8-a9b0-05707bdd2320 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:01.345773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cad33c-05d4-40b8-a9b0-05707bdd2320 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:01.346091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cad33c-05d4-40b8-a9b0-05707bdd2320 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:01.348166 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cad33c-05d4-40b8-a9b0-05707bdd2320 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:01.351615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 339/649] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:16:01.353156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef48c1-f88c-48aa-a51f-1e7143fe3de9 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:01.370480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 311/650] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:01.370480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cad33c-05d4-40b8-a9b0-05707bdd2320 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:01.427516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:01.501547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:01.536096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef48c1-f88c-48aa-a51f-1e7143fe3de9 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:01.541898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cad33c-05d4-40b8-a9b0-05707bdd2320 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:01.598269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfeb1f-2c6a-4f3c-b578-ee15b04b6528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:01.599458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfeb1f-2c6a-4f3c-b578-ee15b04b6528 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:01.599900 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc480ec6-2915-4f29-b479-aebc89daa5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:01.600105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfeb1f-2c6a-4f3c-b578-ee15b04b6528 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:01.600386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfeb1f-2c6a-4f3c-b578-ee15b04b6528 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:01.601266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc480ec6-2915-4f29-b479-aebc89daa5cb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:01.601625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc480ec6-2915-4f29-b479-aebc89daa5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:01.602025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc480ec6-2915-4f29-b479-aebc89daa5cb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:01.616089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc480ec6-2915-4f29-b479-aebc89daa5cb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:01.618634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc480ec6-2915-4f29-b479-aebc89daa5cb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:01.619435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfeb1f-2c6a-4f3c-b578-ee15b04b6528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-9dbfeb1f-2c6a-4f3c-b578-ee15b04b6528 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:01.634087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:01.636080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:01.637501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 340/651] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:16:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:01.640082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfeb1f-2c6a-4f3c-b578-ee15b04b6528 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-cc480ec6-2915-4f29-b479-aebc89daa5cb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:01.655631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:01.657474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:01.658949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 312/652] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:16:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:01.662257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc480ec6-2915-4f29-b479-aebc89daa5cb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:01.811123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfeb1f-2c6a-4f3c-b578-ee15b04b6528 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:01.817365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:01.827494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc480ec6-2915-4f29-b479-aebc89daa5cb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:01.870089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff70b5-4eb8-4767-b13a-bf2dc77fae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:01.871167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff70b5-4eb8-4767-b13a-bf2dc77fae57 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:01.871706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff70b5-4eb8-4767-b13a-bf2dc77fae57 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:01.872199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff70b5-4eb8-4767-b13a-bf2dc77fae57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:01.873230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ff30b-ada5-417b-8587-98ad682bd6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:01.873945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ff30b-ada5-417b-8587-98ad682bd6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:01.874246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ff30b-ada5-417b-8587-98ad682bd6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:01.874503 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ff30b-ada5-417b-8587-98ad682bd6e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:01.875362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ff30b-ada5-417b-8587-98ad682bd6e8 None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=e33d1473077247cc97dcfd99b3da5599) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:01.875728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ff30b-ada5-417b-8587-98ad682bd6e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:01.877268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ff30b-ada5-417b-8587-98ad682bd6e8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:01.893897 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 313/653] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:01 2022] GET /identity/v3/role_assignments?user.id=e33d1473077247cc97dcfd99b3da5599&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:01.893897 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ff30b-ada5-417b-8587-98ad682bd6e8 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:01.900406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff70b5-4eb8-4767-b13a-bf2dc77fae57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:01.904573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff70b5-4eb8-4767-b13a-bf2dc77fae57 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:01.924308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-41ff70b5-4eb8-4767-b13a-bf2dc77fae57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:01.950561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:01.951729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:01.952824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:01.952824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:01.952824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:01.952824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:01.954065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 341/654] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:01.955698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff70b5-4eb8-4767-b13a-bf2dc77fae57 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:02.037841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ff30b-ada5-417b-8587-98ad682bd6e8 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:02.090534 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff70b5-4eb8-4767-b13a-bf2dc77fae57 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:02.090775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac177399-8533-4053-9bad-0ed0fcdc7b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:02.091480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac177399-8533-4053-9bad-0ed0fcdc7b46 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:02.091848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac177399-8533-4053-9bad-0ed0fcdc7b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:02.092127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac177399-8533-4053-9bad-0ed0fcdc7b46 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:02.092898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac177399-8533-4053-9bad-0ed0fcdc7b46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:02.093206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac177399-8533-4053-9bad-0ed0fcdc7b46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:02.094595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac177399-8533-4053-9bad-0ed0fcdc7b46 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:02.110062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 314/655] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:16:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:02.110465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac177399-8533-4053-9bad-0ed0fcdc7b46 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:02.141187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1259e-4dae-40e2-a185-e9dd11e194d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:02.142014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1259e-4dae-40e2-a185-e9dd11e194d2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:02.146407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1259e-4dae-40e2-a185-e9dd11e194d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:02.146702 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1259e-4dae-40e2-a185-e9dd11e194d2 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:02.147605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1259e-4dae-40e2-a185-e9dd11e194d2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:02.147904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1259e-4dae-40e2-a185-e9dd11e194d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:02.149368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1259e-4dae-40e2-a185-e9dd11e194d2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:02.156084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:02.177027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 342/656] 10.222.0.64 () {56 vars in 1139 bytes} [Mon Jun 27 09:16:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:02.178260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1259e-4dae-40e2-a185-e9dd11e194d2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:02.246964 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:02.254091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac177399-8533-4053-9bad-0ed0fcdc7b46 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:02.305928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84916a04-86f1-471e-b60f-a1eb751dcecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:02.307142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84916a04-86f1-471e-b60f-a1eb751dcecb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:02.307427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84916a04-86f1-471e-b60f-a1eb751dcecb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:02.307815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84916a04-86f1-471e-b60f-a1eb751dcecb None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:02.308856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84916a04-86f1-471e-b60f-a1eb751dcecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:02.309165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84916a04-86f1-471e-b60f-a1eb751dcecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:02.310579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84916a04-86f1-471e-b60f-a1eb751dcecb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:02.321975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1259e-4dae-40e2-a185-e9dd11e194d2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:02.331704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 315/657] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:02.332961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84916a04-86f1-471e-b60f-a1eb751dcecb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:02.363682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cee9ad-d688-4fe7-955d-4eed1534b9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:02.364508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cee9ad-d688-4fe7-955d-4eed1534b9db None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:02.364791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cee9ad-d688-4fe7-955d-4eed1534b9db None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:02.365068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cee9ad-d688-4fe7-955d-4eed1534b9db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:02.376842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cee9ad-d688-4fe7-955d-4eed1534b9db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:02.378836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cee9ad-d688-4fe7-955d-4eed1534b9db None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-a6cee9ad-d688-4fe7-955d-4eed1534b9db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:02.399989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:02.401284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:02.402318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:02.402318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:02.402318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:02.402318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:02.405970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 343/658] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:02.406970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cee9ad-d688-4fe7-955d-4eed1534b9db None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:02.473057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:02.497231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84916a04-86f1-471e-b60f-a1eb751dcecb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:02.523189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cee9ad-d688-4fe7-955d-4eed1534b9db None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:02.528862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:02.568973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a47cd-46f8-4dc3-9f75-c4a3d95ed05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:02.570632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a47cd-46f8-4dc3-9f75-c4a3d95ed05e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:02.571429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a47cd-46f8-4dc3-9f75-c4a3d95ed05e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:02.572313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a47cd-46f8-4dc3-9f75-c4a3d95ed05e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:02.592669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6381006-8688-470a-8a9f-3b6f40f02e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:02.594542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6381006-8688-470a-8a9f-3b6f40f02e89 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:02.595346 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6381006-8688-470a-8a9f-3b6f40f02e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:02.596080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6381006-8688-470a-8a9f-3b6f40f02e89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:02.610806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6381006-8688-470a-8a9f-3b6f40f02e89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:02.614388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6381006-8688-470a-8a9f-3b6f40f02e89 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-b6381006-8688-470a-8a9f-3b6f40f02e89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:02.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:02.648321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:02.649328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:02.649328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:02.649328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:02.649328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:02.649328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:02.649328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:02.649328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:02.649328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:02.651424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 344/659] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:02.653332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6381006-8688-470a-8a9f-3b6f40f02e89 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:02.680335 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a47cd-46f8-4dc3-9f75-c4a3d95ed05e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:02.681885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a47cd-46f8-4dc3-9f75-c4a3d95ed05e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:02.789644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:02.816537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 316/660] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:16:02.819025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a47cd-46f8-4dc3-9f75-c4a3d95ed05e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:02.916385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6381006-8688-470a-8a9f-3b6f40f02e89 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:02.952822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a47cd-46f8-4dc3-9f75-c4a3d95ed05e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:02.979160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a375345-29bd-451c-b766-4b67595c2fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:02.980049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375345-29bd-451c-b766-4b67595c2fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:02.980331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375345-29bd-451c-b766-4b67595c2fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:02.980607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375345-29bd-451c-b766-4b67595c2fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:02.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a375345-29bd-451c-b766-4b67595c2fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:02.981655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a375345-29bd-451c-b766-4b67595c2fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:02.983067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a375345-29bd-451c-b766-4b67595c2fe2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:02.991586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08857529-d168-4d72-9b21-3e1b1d87cbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:02.992273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08857529-d168-4d72-9b21-3e1b1d87cbdd None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:02.992575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08857529-d168-4d72-9b21-3e1b1d87cbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:02.992847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08857529-d168-4d72-9b21-3e1b1d87cbdd None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:02.993669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08857529-d168-4d72-9b21-3e1b1d87cbdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:02.993990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08857529-d168-4d72-9b21-3e1b1d87cbdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:02.995509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08857529-d168-4d72-9b21-3e1b1d87cbdd None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:03.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 345/661] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:03.005314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a375345-29bd-451c-b766-4b67595c2fe2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:03.009679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 317/662] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:03.011180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f488be-a5ed-44ca-b183-f54aee4cb5c2 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:03.012514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f488be-a5ed-44ca-b183-f54aee4cb5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:03.012514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f488be-a5ed-44ca-b183-f54aee4cb5c2 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:03.013626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 318/663] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:03.014481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f488be-a5ed-44ca-b183-f54aee4cb5c2 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:03.038305 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:03.072866 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:03.147441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a375345-29bd-451c-b766-4b67595c2fe2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:03.189614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f488be-a5ed-44ca-b183-f54aee4cb5c2 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:03.196264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bc4d9-307e-4577-8c29-14274a379fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:03.197499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bc4d9-307e-4577-8c29-14274a379fbb None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:03.197792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bc4d9-307e-4577-8c29-14274a379fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:03.198256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bc4d9-307e-4577-8c29-14274a379fbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:03.217039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bc4d9-307e-4577-8c29-14274a379fbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:03.219637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bc4d9-307e-4577-8c29-14274a379fbb None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:03.239075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448575e-c882-4c13-a6a3-f5805809103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:03.242347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448575e-c882-4c13-a6a3-f5805809103f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:03.242347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448575e-c882-4c13-a6a3-f5805809103f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:03.242347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448575e-c882-4c13-a6a3-f5805809103f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-df4bc4d9-307e-4577-8c29-14274a379fbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:03.243306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:03.244750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:03.245919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:03.245919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:03.245919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:03.245919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:03.249421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 346/664] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:03.250421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bc4d9-307e-4577-8c29-14274a379fbb None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:03.258203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448575e-c882-4c13-a6a3-f5805809103f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:03.261097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448575e-c882-4c13-a6a3-f5805809103f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0448575e-c882-4c13-a6a3-f5805809103f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:03.294496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:03.296508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 09:16:03.298358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:03.309136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 319/665] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:16:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:03.309136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97df2-78fb-405a-ba69-d744e8a3cb61 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:03.309136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97df2-78fb-405a-ba69-d744e8a3cb61 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:03.309136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97df2-78fb-405a-ba69-d744e8a3cb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:03.336784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a1c97df2-78fb-405a-ba69-d744e8a3cb61 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:03.338883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a1c97df2-78fb-405a-ba69-d744e8a3cb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:03.358889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:03.387819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bc4d9-307e-4577-8c29-14274a379fbb None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:03.434358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2e899-ade7-4534-887d-fe97ad5b3577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:03.435027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2e899-ade7-4534-887d-fe97ad5b3577 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:03.435313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2e899-ade7-4534-887d-fe97ad5b3577 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:03.435623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2e899-ade7-4534-887d-fe97ad5b3577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:03.436563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2e899-ade7-4534-887d-fe97ad5b3577 None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19e0fbe0b5814ad1ae95c65a64b632d6) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:03.436875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2e899-ade7-4534-887d-fe97ad5b3577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:03.438336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c2e899-ade7-4534-887d-fe97ad5b3577 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:03.451684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 347/666] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:03 2022] GET /identity/v3/role_assignments?user.id=19e0fbe0b5814ad1ae95c65a64b632d6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:03.451684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2e899-ade7-4534-887d-fe97ad5b3577 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:03.452605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 320/667] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:03.456614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c97df2-78fb-405a-ba69-d744e8a3cb61 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:03.509168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:03.626107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2e899-ade7-4534-887d-fe97ad5b3577 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:03.649076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c97df2-78fb-405a-ba69-d744e8a3cb61 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:03.719890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cee95-46d2-4de4-8cc6-a5084507663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:03.720155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95beb48-63b9-4a2c-822c-517529316e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:03.721000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cee95-46d2-4de4-8cc6-a5084507663d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:03.721165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95beb48-63b9-4a2c-822c-517529316e4b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:03.721531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cee95-46d2-4de4-8cc6-a5084507663d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:03.721706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95beb48-63b9-4a2c-822c-517529316e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:03.722023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cee95-46d2-4de4-8cc6-a5084507663d None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:03.722234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95beb48-63b9-4a2c-822c-517529316e4b None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:03.723131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95beb48-63b9-4a2c-822c-517529316e4b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:03.723562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cee95-46d2-4de4-8cc6-a5084507663d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:03.724153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cee95-46d2-4de4-8cc6-a5084507663d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:03.725589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95beb48-63b9-4a2c-822c-517529316e4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:03.726649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cee95-46d2-4de4-8cc6-a5084507663d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:03.726828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95beb48-63b9-4a2c-822c-517529316e4b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:03.742589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:03.750347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 348/668] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:03.751172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cee95-46d2-4de4-8cc6-a5084507663d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:03.763119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 321/669] 10.222.0.64 () {56 vars in 1137 bytes} [Mon Jun 27 09:16:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:03.764555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95beb48-63b9-4a2c-822c-517529316e4b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:03.923042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:03.927886 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cee95-46d2-4de4-8cc6-a5084507663d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:03.942252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95beb48-63b9-4a2c-822c-517529316e4b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:03.980871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc39c3-89e2-4b37-8844-84d356a24b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:03.981628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b6a9-33e7-4572-9d76-bd1a90daa32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:03.981870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc39c3-89e2-4b37-8844-84d356a24b4b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:03.982297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc39c3-89e2-4b37-8844-84d356a24b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:03.982679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc39c3-89e2-4b37-8844-84d356a24b4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:03.983308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b6a9-33e7-4572-9d76-bd1a90daa32b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:03.983856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b6a9-33e7-4572-9d76-bd1a90daa32b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:03.984276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b6a9-33e7-4572-9d76-bd1a90daa32b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:03.991212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc39c3-89e2-4b37-8844-84d356a24b4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:03.993016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc39c3-89e2-4b37-8844-84d356a24b4b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:03.997574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b6a9-33e7-4572-9d76-bd1a90daa32b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-7e08b6a9-33e7-4572-9d76-bd1a90daa32b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:04.006674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:04.007930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:04.008952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:04.011519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 349/670] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:16:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:04.012588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b6a9-33e7-4572-9d76-bd1a90daa32b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-abcc39c3-89e2-4b37-8844-84d356a24b4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:04.016915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:04.018081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:04.019750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:04.019750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:04.019750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:04.019750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:04.019750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:04.019750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:04.019750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:04.019750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:04.021790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 322/671] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:04.022752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc39c3-89e2-4b37-8844-84d356a24b4b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:04.093608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:04.150659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b6a9-33e7-4572-9d76-bd1a90daa32b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:04.163439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc39c3-89e2-4b37-8844-84d356a24b4b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:04.202460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35396eac-d334-420b-a53b-06429aafd213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:04.204074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35396eac-d334-420b-a53b-06429aafd213 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:04.204381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35396eac-d334-420b-a53b-06429aafd213 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:04.204665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35396eac-d334-420b-a53b-06429aafd213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:04.226591 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6cc5a-219f-4515-9779-834112293d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:04.228167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6cc5a-219f-4515-9779-834112293d9e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:04.228925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6cc5a-219f-4515-9779-834112293d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:04.229681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6cc5a-219f-4515-9779-834112293d9e None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:04.231243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6cc5a-219f-4515-9779-834112293d9e None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:04.231926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6cc5a-219f-4515-9779-834112293d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:04.234875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6cc5a-219f-4515-9779-834112293d9e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:04.246707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 323/672] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:16:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:04.251469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6cc5a-219f-4515-9779-834112293d9e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:04.269095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:04.322461 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35396eac-d334-420b-a53b-06429aafd213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:04.325151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35396eac-d334-420b-a53b-06429aafd213 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:04.385182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6cc5a-219f-4515-9779-834112293d9e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:04.421762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:04.435316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86284403-26b2-45de-81d3-98aa89b82bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:04.437008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86284403-26b2-45de-81d3-98aa89b82bfb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:04.437537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86284403-26b2-45de-81d3-98aa89b82bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:04.438026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86284403-26b2-45de-81d3-98aa89b82bfb None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:04.439307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86284403-26b2-45de-81d3-98aa89b82bfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:04.439843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86284403-26b2-45de-81d3-98aa89b82bfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:04.441905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86284403-26b2-45de-81d3-98aa89b82bfb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:04.459568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 324/673] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:04.460919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86284403-26b2-45de-81d3-98aa89b82bfb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:04.489927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 350/674] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:16:04.491195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35396eac-d334-420b-a53b-06429aafd213 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:04.548893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:04.582989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86284403-26b2-45de-81d3-98aa89b82bfb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:04.639571 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a388c46-d611-44c0-a06f-19ff9add3c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:04.640419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388c46-d611-44c0-a06f-19ff9add3c86 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:04.640700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388c46-d611-44c0-a06f-19ff9add3c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:04.640978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388c46-d611-44c0-a06f-19ff9add3c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:04.653523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35396eac-d334-420b-a53b-06429aafd213 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:04.654829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a388c46-d611-44c0-a06f-19ff9add3c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:04.656382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a388c46-d611-44c0-a06f-19ff9add3c86 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-7a388c46-d611-44c0-a06f-19ff9add3c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:04.676545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:04.677563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:04.678548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:04.678548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:04.678548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:04.678548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:04.678548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 325/675] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:04.678847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a388c46-d611-44c0-a06f-19ff9add3c86 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:04.706706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421c549-9b6d-4e4b-9627-74968f30d1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:04.708311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421c549-9b6d-4e4b-9627-74968f30d1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:04.708843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421c549-9b6d-4e4b-9627-74968f30d1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:04.709317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421c549-9b6d-4e4b-9627-74968f30d1d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:04.710415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421c549-9b6d-4e4b-9627-74968f30d1d5 None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=2b87f03cae6c4ca98a82dd5d981ed224) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:04.710927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421c549-9b6d-4e4b-9627-74968f30d1d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:04.713241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421c549-9b6d-4e4b-9627-74968f30d1d5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:04.727958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 351/676] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:04 2022] GET /identity/v3/role_assignments?user.id=2b87f03cae6c4ca98a82dd5d981ed224&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:04.730579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad19ad3-95e4-4d7c-9e6a-165e0e1fadd4 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:04.731552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad19ad3-95e4-4d7c-9e6a-165e0e1fadd4 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:04.732095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad19ad3-95e4-4d7c-9e6a-165e0e1fadd4 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:04.734237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 352/677] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:04.735857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad19ad3-95e4-4d7c-9e6a-165e0e1fadd4 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:04.791545 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:04.858580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a388c46-d611-44c0-a06f-19ff9add3c86 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:04.896147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:04.912540 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6879c-05d1-4b2c-abb1-13db18fa5274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:04.913797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6879c-05d1-4b2c-abb1-13db18fa5274 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:04.914264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6879c-05d1-4b2c-abb1-13db18fa5274 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:04.914726 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6879c-05d1-4b2c-abb1-13db18fa5274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:04.929864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6879c-05d1-4b2c-abb1-13db18fa5274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:04.931960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6879c-05d1-4b2c-abb1-13db18fa5274 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-22f6879c-05d1-4b2c-abb1-13db18fa5274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:04.953720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:04.954711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:04.956180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:04.956180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:04.956180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:04.956180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:04.962113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 326/678] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:04.962113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b207e-00f6-44b1-9f17-fd1c899c37d8 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:04.962381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad19ad3-95e4-4d7c-9e6a-165e0e1fadd4 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:04.962808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b207e-00f6-44b1-9f17-fd1c899c37d8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:04.962808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b207e-00f6-44b1-9f17-fd1c899c37d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:04.983636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-621b207e-00f6-44b1-9f17-fd1c899c37d8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:04.984876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-621b207e-00f6-44b1-9f17-fd1c899c37d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:05.018913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556fe68-5088-4a9b-92a4-435d50e6ceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:05.019437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556fe68-5088-4a9b-92a4-435d50e6ceb5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:05.019804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556fe68-5088-4a9b-92a4-435d50e6ceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:05.020082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556fe68-5088-4a9b-92a4-435d50e6ceb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:05.020870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556fe68-5088-4a9b-92a4-435d50e6ceb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:05.021170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556fe68-5088-4a9b-92a4-435d50e6ceb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:05.022562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556fe68-5088-4a9b-92a4-435d50e6ceb5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:05.046903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 353/679] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:05.053664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556fe68-5088-4a9b-92a4-435d50e6ceb5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:05.121090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 327/680] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:05.124399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b207e-00f6-44b1-9f17-fd1c899c37d8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:05.237194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:05.282036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556fe68-5088-4a9b-92a4-435d50e6ceb5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:05.313501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b207e-00f6-44b1-9f17-fd1c899c37d8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:05.317379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:05.341543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bb554-c438-4713-be40-8b01444749d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:05.342824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bb554-c438-4713-be40-8b01444749d0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:05.343303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bb554-c438-4713-be40-8b01444749d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:05.344039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308bb554-c438-4713-be40-8b01444749d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:05.345311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bb554-c438-4713-be40-8b01444749d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:05.345826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bb554-c438-4713-be40-8b01444749d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:05.347502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308bb554-c438-4713-be40-8b01444749d0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:05.354428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65016728-64f3-4510-b52d-ab3879096cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:05.355244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65016728-64f3-4510-b52d-ab3879096cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:05.355556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65016728-64f3-4510-b52d-ab3879096cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:05.355841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65016728-64f3-4510-b52d-ab3879096cf1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:05.361981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 354/681] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:05.363266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bb554-c438-4713-be40-8b01444749d0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:05.369500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65016728-64f3-4510-b52d-ab3879096cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:05.372308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65016728-64f3-4510-b52d-ab3879096cf1 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-65016728-64f3-4510-b52d-ab3879096cf1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:05.392247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:05.394024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:05.395018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 328/682] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:05.395635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65016728-64f3-4510-b52d-ab3879096cf1 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:05.490265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308bb554-c438-4713-be40-8b01444749d0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:05.533786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65016728-64f3-4510-b52d-ab3879096cf1 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:05.535113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c283de-9526-4aed-85b8-6d921c1e4691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:05.536140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c283de-9526-4aed-85b8-6d921c1e4691 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:05.536423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c283de-9526-4aed-85b8-6d921c1e4691 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:05.536700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c283de-9526-4aed-85b8-6d921c1e4691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:05.551413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c283de-9526-4aed-85b8-6d921c1e4691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:05.553361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c283de-9526-4aed-85b8-6d921c1e4691 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:05.557836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:05.575794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471ff4e-9104-432d-9a08-68e0c2605488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:05.575794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471ff4e-9104-432d-9a08-68e0c2605488 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:05.575794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471ff4e-9104-432d-9a08-68e0c2605488 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:05.575794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471ff4e-9104-432d-9a08-68e0c2605488 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-f0c283de-9526-4aed-85b8-6d921c1e4691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:05.579045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:05.580118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:05.581058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:05.581058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:05.581058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:05.581058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:05.583546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 355/683] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:05.583804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471ff4e-9104-432d-9a08-68e0c2605488 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:05.584891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c283de-9526-4aed-85b8-6d921c1e4691 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:05.585418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471ff4e-9104-432d-9a08-68e0c2605488 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-e471ff4e-9104-432d-9a08-68e0c2605488 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:05.601377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:05.602368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:05.603590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:05.603590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:05.603590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:05.603590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:05.603590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:05.603590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:05.603590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:05.603590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:05.607594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 329/684] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:05.608617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471ff4e-9104-432d-9a08-68e0c2605488 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:05.635823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:05.717757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c283de-9526-4aed-85b8-6d921c1e4691 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:05.741205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471ff4e-9104-432d-9a08-68e0c2605488 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:05.761433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff971240-4289-4b80-9ff6-f549b869bb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:05.762839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff971240-4289-4b80-9ff6-f549b869bb98 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:05.763572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff971240-4289-4b80-9ff6-f549b869bb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:05.763965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff971240-4289-4b80-9ff6-f549b869bb98 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:05.764892 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff971240-4289-4b80-9ff6-f549b869bb98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:05.765281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff971240-4289-4b80-9ff6-f549b869bb98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:05.767309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff971240-4289-4b80-9ff6-f549b869bb98 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:05.793185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 356/685] 10.222.0.64 () {56 vars in 1135 bytes} [Mon Jun 27 09:16:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:05.794386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff971240-4289-4b80-9ff6-f549b869bb98 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:05.794826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2ff7f-98ea-460c-9d96-22f84ebbbeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:05.795713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2ff7f-98ea-460c-9d96-22f84ebbbeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:05.796106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2ff7f-98ea-460c-9d96-22f84ebbbeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:05.796481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2ff7f-98ea-460c-9d96-22f84ebbbeb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:05.797387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2ff7f-98ea-460c-9d96-22f84ebbbeb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:05.797761 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2ff7f-98ea-460c-9d96-22f84ebbbeb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:05.801389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2ff7f-98ea-460c-9d96-22f84ebbbeb3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:05.817715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 330/686] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:05.819259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2ff7f-98ea-460c-9d96-22f84ebbbeb3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:05.861610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:05.920704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff971240-4289-4b80-9ff6-f549b869bb98 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:05.936155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:05.953707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2ff7f-98ea-460c-9d96-22f84ebbbeb3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:05.968304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5537715-c4bd-4495-a3ff-732d9b6be15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:05.969399 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5537715-c4bd-4495-a3ff-732d9b6be15f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:05.969704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5537715-c4bd-4495-a3ff-732d9b6be15f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:05.969988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5537715-c4bd-4495-a3ff-732d9b6be15f None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:05.970945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5537715-c4bd-4495-a3ff-732d9b6be15f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:05.971815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5537715-c4bd-4495-a3ff-732d9b6be15f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:05.973331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5537715-c4bd-4495-a3ff-732d9b6be15f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:05.992557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 357/687] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:05.992557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5537715-c4bd-4495-a3ff-732d9b6be15f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:05.997683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c48dbf-e707-4ed2-916e-88177492e8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:05.998352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c48dbf-e707-4ed2-916e-88177492e8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:05.998639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c48dbf-e707-4ed2-916e-88177492e8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:05.998919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c48dbf-e707-4ed2-916e-88177492e8c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:06.109006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c48dbf-e707-4ed2-916e-88177492e8c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:06.110457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c48dbf-e707-4ed2-916e-88177492e8c4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:06.130067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5537715-c4bd-4495-a3ff-732d9b6be15f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:06.145194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:06.171648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b87dd-b4af-4e1b-9aac-fa1e8a5cf610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:06.173051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b87dd-b4af-4e1b-9aac-fa1e8a5cf610 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:06.173533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b87dd-b4af-4e1b-9aac-fa1e8a5cf610 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:06.174336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b87dd-b4af-4e1b-9aac-fa1e8a5cf610 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:06.176344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b87dd-b4af-4e1b-9aac-fa1e8a5cf610 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:06.179616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b87dd-b4af-4e1b-9aac-fa1e8a5cf610 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:06.211673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:06.230590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 331/688] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:16:06.232080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac79fa-cc9f-49d8-b677-c95818945999 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:06.232381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac79fa-cc9f-49d8-b677-c95818945999 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:06.232659 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac79fa-cc9f-49d8-b677-c95818945999 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:06.235313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 332/689] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:06.235313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac79fa-cc9f-49d8-b677-c95818945999 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:06.293634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 358/690] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:16:05 2022] POST /identity/v3/users => generated 254 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:16:06.294653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b87dd-b4af-4e1b-9aac-fa1e8a5cf610 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:06.376539 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac79fa-cc9f-49d8-b677-c95818945999 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:06.421084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17064693-9b71-4dd1-9651-3ecb3ffbf5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:06.422611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17064693-9b71-4dd1-9651-3ecb3ffbf5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:06.423089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17064693-9b71-4dd1-9651-3ecb3ffbf5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:06.423601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17064693-9b71-4dd1-9651-3ecb3ffbf5a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:06.424885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17064693-9b71-4dd1-9651-3ecb3ffbf5a1 None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=8c8abb981d1b4c82a43b5a3372f98398) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:06.425486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17064693-9b71-4dd1-9651-3ecb3ffbf5a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:06.427197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17064693-9b71-4dd1-9651-3ecb3ffbf5a1 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:06.435600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 333/691] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:06 2022] GET /identity/v3/role_assignments?user.id=8c8abb981d1b4c82a43b5a3372f98398&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:06.437401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f7716-0230-4c33-b7f1-638b41b7dc02 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:06.438009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f7716-0230-4c33-b7f1-638b41b7dc02 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:06.438496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f7716-0230-4c33-b7f1-638b41b7dc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:06.439580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b87dd-b4af-4e1b-9aac-fa1e8a5cf610 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:06.458912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d29f7716-0230-4c33-b7f1-638b41b7dc02 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:06.460794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d29f7716-0230-4c33-b7f1-638b41b7dc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:06.482276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae232e24-3870-4d78-b289-f34bde8f54c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:06.483265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae232e24-3870-4d78-b289-f34bde8f54c4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:06.483710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae232e24-3870-4d78-b289-f34bde8f54c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:06.484109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae232e24-3870-4d78-b289-f34bde8f54c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:06.496553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae232e24-3870-4d78-b289-f34bde8f54c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:06.498503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae232e24-3870-4d78-b289-f34bde8f54c4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-ae232e24-3870-4d78-b289-f34bde8f54c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:06.518655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:06.519658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:06.520585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:06.520585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:06.520585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:06.520585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:06.520585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 359/692] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:06.525823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:06.527188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae232e24-3870-4d78-b289-f34bde8f54c4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:06.559497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 334/693] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:06.564050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:06.585507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f7716-0230-4c33-b7f1-638b41b7dc02 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:06.655667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae232e24-3870-4d78-b289-f34bde8f54c4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:06.712609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f7716-0230-4c33-b7f1-638b41b7dc02 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:06.717284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa108d-864d-4db8-a5a8-e35c008cb4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:06.717972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa108d-864d-4db8-a5a8-e35c008cb4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:06.718261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa108d-864d-4db8-a5a8-e35c008cb4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:06.718561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa108d-864d-4db8-a5a8-e35c008cb4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:06.719304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa108d-864d-4db8-a5a8-e35c008cb4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:06.719645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa108d-864d-4db8-a5a8-e35c008cb4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:06.721055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa108d-864d-4db8-a5a8-e35c008cb4a2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:06.737488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 360/694] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:06.746377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa108d-864d-4db8-a5a8-e35c008cb4a2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:06.767654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126be76-1f0d-4499-b936-910009c6aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:06.769047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126be76-1f0d-4499-b936-910009c6aabd None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:06.769365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126be76-1f0d-4499-b936-910009c6aabd None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:06.769641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126be76-1f0d-4499-b936-910009c6aabd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:06.783192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126be76-1f0d-4499-b936-910009c6aabd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-5126be76-1f0d-4499-b936-910009c6aabd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:06.791289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:06.792593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:06.793644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:06.797333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 335/695] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:06.798565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126be76-1f0d-4499-b936-910009c6aabd None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:06.844482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:06.862961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:06.908201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa108d-864d-4db8-a5a8-e35c008cb4a2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:06.949493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126be76-1f0d-4499-b936-910009c6aabd None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:06.975727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7bd5-f89d-40e6-b9fb-0a135ec21a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:06.977112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7bd5-f89d-40e6-b9fb-0a135ec21a3d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:06.977932 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7bd5-f89d-40e6-b9fb-0a135ec21a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:06.979517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7bd5-f89d-40e6-b9fb-0a135ec21a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:06.992480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed7bd5-f89d-40e6-b9fb-0a135ec21a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:06.994593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed7bd5-f89d-40e6-b9fb-0a135ec21a3d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:07.024579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b9c3b-2d51-430a-a733-9cb06f415d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:07.025916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b9c3b-2d51-430a-a733-9cb06f415d87 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:07.026390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b9c3b-2d51-430a-a733-9cb06f415d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:07.026908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b9c3b-2d51-430a-a733-9cb06f415d87 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:07.027883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b9c3b-2d51-430a-a733-9cb06f415d87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:07.028375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b9c3b-2d51-430a-a733-9cb06f415d87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:07.030023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b9c3b-2d51-430a-a733-9cb06f415d87 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-80ed7bd5-f89d-40e6-b9fb-0a135ec21a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:07.030250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:07.031111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:07.032100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:07.032100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:07.032100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:07.032100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:07.032100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:07.032100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:07.032100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:07.032100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:07.034748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 361/696] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:07.040720 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7bd5-f89d-40e6-b9fb-0a135ec21a3d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:07.061269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 336/697] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:16:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:07.067609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b9c3b-2d51-430a-a733-9cb06f415d87 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:07.224614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7bd5-f89d-40e6-b9fb-0a135ec21a3d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:07.232789 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:07.236317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b9c3b-2d51-430a-a733-9cb06f415d87 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:07.278605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4bfa6-4c70-4d74-9b54-1cb82c729ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:07.279303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4bfa6-4c70-4d74-9b54-1cb82c729ddc None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:07.279627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4bfa6-4c70-4d74-9b54-1cb82c729ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:07.279907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4bfa6-4c70-4d74-9b54-1cb82c729ddc None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:07.280717 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4bfa6-4c70-4d74-9b54-1cb82c729ddc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:07.281030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4bfa6-4c70-4d74-9b54-1cb82c729ddc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:07.282758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf67d5d-a03e-4675-bd7e-6255b94e39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:07.283129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac4bfa6-4c70-4d74-9b54-1cb82c729ddc None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:07.290081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf67d5d-a03e-4675-bd7e-6255b94e39d8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:07.290736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf67d5d-a03e-4675-bd7e-6255b94e39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:07.291307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf67d5d-a03e-4675-bd7e-6255b94e39d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:07.304222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 362/698] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:07.304222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66ce57-91b8-4491-b419-125934aa68ef None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:07.304222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66ce57-91b8-4491-b419-125934aa68ef None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:07.304222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66ce57-91b8-4491-b419-125934aa68ef None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:07.304222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 363/699] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:07.306072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66ce57-91b8-4491-b419-125934aa68ef None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:07.309411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf67d5d-a03e-4675-bd7e-6255b94e39d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:07.309857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:07.311470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf67d5d-a03e-4675-bd7e-6255b94e39d8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-bcf67d5d-a03e-4675-bd7e-6255b94e39d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:07.335043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:07.336277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:07.337463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:07.337463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:07.337463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:07.337463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:07.339613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 337/700] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:07.341260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3d2fd-233e-44f5-9bcd-1ee5930d73ac None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:07.342917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3d2fd-233e-44f5-9bcd-1ee5930d73ac None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:07.343200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3d2fd-233e-44f5-9bcd-1ee5930d73ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:07.362968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-90b3d2fd-233e-44f5-9bcd-1ee5930d73ac None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:07.364236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-90b3d2fd-233e-44f5-9bcd-1ee5930d73ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:07.448455 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66ce57-91b8-4491-b419-125934aa68ef None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:07.450567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 338/701] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:07.452517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3d2fd-233e-44f5-9bcd-1ee5930d73ac None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:07.495578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b13656-c713-44dd-b51f-ff31feb61bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:07.496546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b13656-c713-44dd-b51f-ff31feb61bfe None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:07.497035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b13656-c713-44dd-b51f-ff31feb61bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:07.497672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b13656-c713-44dd-b51f-ff31feb61bfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:07.498706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b13656-c713-44dd-b51f-ff31feb61bfe None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=70b499a32516486d9a2bed34f7e66b76) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:07.499599 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b13656-c713-44dd-b51f-ff31feb61bfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:07.501568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b13656-c713-44dd-b51f-ff31feb61bfe None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:07.509953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 364/702] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:07 2022] GET /identity/v3/role_assignments?user.id=70b499a32516486d9a2bed34f7e66b76&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:07.511088 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b13656-c713-44dd-b51f-ff31feb61bfe None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:07.644370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:07.670451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:07.703644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3d2fd-233e-44f5-9bcd-1ee5930d73ac None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:07.760756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b13656-c713-44dd-b51f-ff31feb61bfe None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:07.765419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649bb49-02fb-4296-82e0-61867dd17451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:07.766515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649bb49-02fb-4296-82e0-61867dd17451 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:07.767039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649bb49-02fb-4296-82e0-61867dd17451 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:07.767552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649bb49-02fb-4296-82e0-61867dd17451 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:07.768570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649bb49-02fb-4296-82e0-61867dd17451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:07.769130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649bb49-02fb-4296-82e0-61867dd17451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:07.770756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649bb49-02fb-4296-82e0-61867dd17451 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:07.786991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 339/703] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:07.792497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649bb49-02fb-4296-82e0-61867dd17451 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:07.818898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cad02b-1967-4ffe-8b4e-fe86300c359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:07.819759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad02b-1967-4ffe-8b4e-fe86300c359f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:07.820077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad02b-1967-4ffe-8b4e-fe86300c359f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:07.820350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad02b-1967-4ffe-8b4e-fe86300c359f None admin] PATH_INFO: `/v3/services/network` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:07.820975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cad02b-1967-4ffe-8b4e-fe86300c359f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:07.822409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cad02b-1967-4ffe-8b4e-fe86300c359f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-e2cad02b-1967-4ffe-8b4e-fe86300c359f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:07.842350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:07.844103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:07.845083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 365/704] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:07 2022] GET /identity/v3/services/network => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:07.850294 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cad02b-1967-4ffe-8b4e-fe86300c359f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:07.981625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649bb49-02fb-4296-82e0-61867dd17451 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:08.022636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cad02b-1967-4ffe-8b4e-fe86300c359f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:08.023932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf0175-35c0-492b-8179-5c2820f7a1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:08.024801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf0175-35c0-492b-8179-5c2820f7a1ca None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:08.025092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf0175-35c0-492b-8179-5c2820f7a1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:08.025627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf0175-35c0-492b-8179-5c2820f7a1ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:08.036684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf0175-35c0-492b-8179-5c2820f7a1ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:08.038600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf0175-35c0-492b-8179-5c2820f7a1ca None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-1eaf0175-35c0-492b-8179-5c2820f7a1ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:08.066339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:08.067374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:08.068375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:08.070306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:08.070435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 340/705] 10.222.0.64 () {56 vars in 1093 bytes} [Mon Jun 27 09:16:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:08.075106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf0175-35c0-492b-8179-5c2820f7a1ca None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:08.079867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:08.081962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d075ed-e71b-4abf-a242-51ea59331dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:08.084489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d075ed-e71b-4abf-a242-51ea59331dca None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:08.096798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d075ed-e71b-4abf-a242-51ea59331dca None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:08.096798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d075ed-e71b-4abf-a242-51ea59331dca None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:08.096798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d075ed-e71b-4abf-a242-51ea59331dca None admin] RBAC: Adding query filter params (name=network) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:08.096798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d075ed-e71b-4abf-a242-51ea59331dca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:08.096798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d075ed-e71b-4abf-a242-51ea59331dca None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:08.098378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 366/706] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:08.105267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d075ed-e71b-4abf-a242-51ea59331dca None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:08.212350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf0175-35c0-492b-8179-5c2820f7a1ca None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:08.233855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d075ed-e71b-4abf-a242-51ea59331dca None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:08.260212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee1f34-14c2-4773-8f61-19a6eb980835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:08.261302 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee1f34-14c2-4773-8f61-19a6eb980835 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:08.261825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee1f34-14c2-4773-8f61-19a6eb980835 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:08.262321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee1f34-14c2-4773-8f61-19a6eb980835 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:08.263405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee1f34-14c2-4773-8f61-19a6eb980835 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:08.264004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee1f34-14c2-4773-8f61-19a6eb980835 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:08.265850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee1f34-14c2-4773-8f61-19a6eb980835 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:08.302058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 341/707] 10.222.0.64 () {56 vars in 1133 bytes} [Mon Jun 27 09:16:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:08.307520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee1f34-14c2-4773-8f61-19a6eb980835 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:08.312065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ca02d-a617-4992-b6e2-5003c0abaa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:08.315505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ca02d-a617-4992-b6e2-5003c0abaa4b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:08.321503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ca02d-a617-4992-b6e2-5003c0abaa4b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:08.322126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ca02d-a617-4992-b6e2-5003c0abaa4b None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:08.323121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ca02d-a617-4992-b6e2-5003c0abaa4b None admin] RBAC: Adding query filter params (type=network) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:08.323687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ca02d-a617-4992-b6e2-5003c0abaa4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:08.325390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ca02d-a617-4992-b6e2-5003c0abaa4b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:08.342016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 367/708] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:08.399972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:08.448092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee1f34-14c2-4773-8f61-19a6eb980835 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:08.490258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660f360-7064-40ff-9cc6-1225d620c7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:08.491866 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660f360-7064-40ff-9cc6-1225d620c7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:08.492391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660f360-7064-40ff-9cc6-1225d620c7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:08.493113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660f360-7064-40ff-9cc6-1225d620c7b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:08.579808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660f360-7064-40ff-9cc6-1225d620c7b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:08.581759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660f360-7064-40ff-9cc6-1225d620c7b8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:08.666902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:08.774391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 342/709] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:16:08.780994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ca02d-a617-4992-b6e2-5003c0abaa4b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:08.877975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:08.930174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ca02d-a617-4992-b6e2-5003c0abaa4b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:08.946465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77029aba-aacc-440b-b065-974fdd209caa None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:08.946802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77029aba-aacc-440b-b065-974fdd209caa None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:08.947087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77029aba-aacc-440b-b065-974fdd209caa None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:08.948830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 343/710] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:08.954325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5a2c3-bb62-4b3e-b7d7-bddae4025180 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:08.954652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5a2c3-bb62-4b3e-b7d7-bddae4025180 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:08.954899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5a2c3-bb62-4b3e-b7d7-bddae4025180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:08.976605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8f0e9-cd8d-4141-ac87-ebbe88f1c2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:08.978402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f0e9-cd8d-4141-ac87-ebbe88f1c2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:08.978402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f0e9-cd8d-4141-ac87-ebbe88f1c2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:08.978402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f0e9-cd8d-4141-ac87-ebbe88f1c2e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:08.991717 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8f0e9-cd8d-4141-ac87-ebbe88f1c2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:08.993908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8f0e9-cd8d-4141-ac87-ebbe88f1c2e0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:09.008696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-aaa5a2c3-bb62-4b3e-b7d7-bddae4025180 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:09.009939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-aaa5a2c3-bb62-4b3e-b7d7-bddae4025180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-5fc8f0e9-cd8d-4141-ac87-ebbe88f1c2e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:09.014047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:09.016039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:09.017194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:09.017194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:09.017194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:09.017194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:09.017194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 368/711] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:09.027433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8f0e9-cd8d-4141-ac87-ebbe88f1c2e0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:09.146694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8f0e9-cd8d-4141-ac87-ebbe88f1c2e0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:09.160620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:09.187878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f941b-9a03-4d9d-b461-e06c3b86d9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:09.188760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f941b-9a03-4d9d-b461-e06c3b86d9fa None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:09.189176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f941b-9a03-4d9d-b461-e06c3b86d9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:09.189557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f941b-9a03-4d9d-b461-e06c3b86d9fa None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:09.190507 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f941b-9a03-4d9d-b461-e06c3b86d9fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:09.190917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f941b-9a03-4d9d-b461-e06c3b86d9fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:09.192543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f941b-9a03-4d9d-b461-e06c3b86d9fa None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:09.197830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 344/712] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:09.199233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac51ba-7153-4f22-9fa4-f62ab5396776 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:09.199541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac51ba-7153-4f22-9fa4-f62ab5396776 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:09.199830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac51ba-7153-4f22-9fa4-f62ab5396776 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:09.201385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 345/713] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:09.207548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 369/714] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:09.214180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac51ba-7153-4f22-9fa4-f62ab5396776 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:09.218441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5383b7d-4ae4-4c47-bdb3-434e4b05cdcc None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:09.218824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5383b7d-4ae4-4c47-bdb3-434e4b05cdcc None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:09.219162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5383b7d-4ae4-4c47-bdb3-434e4b05cdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:09.241973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d5383b7d-4ae4-4c47-bdb3-434e4b05cdcc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:09.243585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d5383b7d-4ae4-4c47-bdb3-434e4b05cdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:09.333023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 370/715] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:09.334252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5383b7d-4ae4-4c47-bdb3-434e4b05cdcc None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:09.417400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:09.444106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac51ba-7153-4f22-9fa4-f62ab5396776 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:09.517670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834a5e4-6990-4921-89b6-9a7c8cb035ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:09.519602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834a5e4-6990-4921-89b6-9a7c8cb035ac None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:09.520110 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834a5e4-6990-4921-89b6-9a7c8cb035ac None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:09.520582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834a5e4-6990-4921-89b6-9a7c8cb035ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:09.533500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834a5e4-6990-4921-89b6-9a7c8cb035ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:09.535383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:09.538276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834a5e4-6990-4921-89b6-9a7c8cb035ac None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:09.541217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5383b7d-4ae4-4c47-bdb3-434e4b05cdcc None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-7834a5e4-6990-4921-89b6-9a7c8cb035ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:09.570443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:09.572094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:09.574375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:09.574375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:09.574375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:09.574375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:09.574375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:09.574375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:09.574375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:09.574375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:09.585515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 346/716] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:09.587924 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834a5e4-6990-4921-89b6-9a7c8cb035ac None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:09.612372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9017c-f588-45eb-a09a-52fcbce7d4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:09.613672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9017c-f588-45eb-a09a-52fcbce7d4cb None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:09.614146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9017c-f588-45eb-a09a-52fcbce7d4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:09.614614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9017c-f588-45eb-a09a-52fcbce7d4cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:09.630240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9017c-f588-45eb-a09a-52fcbce7d4cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-9dc9017c-f588-45eb-a09a-52fcbce7d4cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:09.639992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:09.641839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:09.644554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:09.646693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 371/717] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:16:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:09.647730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9017c-f588-45eb-a09a-52fcbce7d4cb None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:09.803898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834a5e4-6990-4921-89b6-9a7c8cb035ac None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:09.846192 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9017c-f588-45eb-a09a-52fcbce7d4cb None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:09.882447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0334468-376d-4dfa-94f9-6ea859607887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:09.883326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0334468-376d-4dfa-94f9-6ea859607887 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:09.883652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0334468-376d-4dfa-94f9-6ea859607887 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:09.883937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0334468-376d-4dfa-94f9-6ea859607887 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:09.884640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0334468-376d-4dfa-94f9-6ea859607887 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:09.886086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0334468-376d-4dfa-94f9-6ea859607887 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:09.903186 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f9758-677c-4978-872f-e84332aa1fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-b0334468-376d-4dfa-94f9-6ea859607887 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:09.903451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:09.905648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:09.907498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 347/718] 10.222.0.64 () {56 vars in 1129 bytes} [Mon Jun 27 09:16:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:09.912316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0334468-376d-4dfa-94f9-6ea859607887 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:09.914107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f9758-677c-4978-872f-e84332aa1fce None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:09.914826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f9758-677c-4978-872f-e84332aa1fce None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:09.916172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f9758-677c-4978-872f-e84332aa1fce None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:09.918246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f9758-677c-4978-872f-e84332aa1fce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:09.918823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f9758-677c-4978-872f-e84332aa1fce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:09.920557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f9758-677c-4978-872f-e84332aa1fce None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:09.933870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:09.943183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 372/719] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:09.946574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f9758-677c-4978-872f-e84332aa1fce None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:09.997666 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:10.062746 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0334468-376d-4dfa-94f9-6ea859607887 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:10.107039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960ee94-1d73-4c7e-8088-f0f6253d1251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:10.108550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960ee94-1d73-4c7e-8088-f0f6253d1251 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:10.109331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960ee94-1d73-4c7e-8088-f0f6253d1251 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:10.109818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960ee94-1d73-4c7e-8088-f0f6253d1251 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:10.110888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960ee94-1d73-4c7e-8088-f0f6253d1251 None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:10.111663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960ee94-1d73-4c7e-8088-f0f6253d1251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:10.113704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960ee94-1d73-4c7e-8088-f0f6253d1251 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:10.118248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f9758-677c-4978-872f-e84332aa1fce None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:10.124934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 348/720] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:16:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:10.126296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960ee94-1d73-4c7e-8088-f0f6253d1251 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:10.182794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313d857-5100-47b4-9271-a5ed5b2493cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:10.183754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d857-5100-47b4-9271-a5ed5b2493cd None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:10.183982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d857-5100-47b4-9271-a5ed5b2493cd None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:10.184261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d857-5100-47b4-9271-a5ed5b2493cd None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:10.185022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313d857-5100-47b4-9271-a5ed5b2493cd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:10.185336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313d857-5100-47b4-9271-a5ed5b2493cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:10.186904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313d857-5100-47b4-9271-a5ed5b2493cd None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:10.192854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 373/721] 10.222.0.64 () {56 vars in 1124 bytes} [Mon Jun 27 09:16:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:10.193871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313d857-5100-47b4-9271-a5ed5b2493cd None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:10.231519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:10.261689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960ee94-1d73-4c7e-8088-f0f6253d1251 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:10.309370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a18d-6560-48e1-8d93-6a257842291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:10.310425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a18d-6560-48e1-8d93-6a257842291b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:10.310771 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a18d-6560-48e1-8d93-6a257842291b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:10.311137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a18d-6560-48e1-8d93-6a257842291b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:10.312344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604a18d-6560-48e1-8d93-6a257842291b None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19e0fbe0b5814ad1ae95c65a64b632d6) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:10.312731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604a18d-6560-48e1-8d93-6a257842291b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:10.315437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604a18d-6560-48e1-8d93-6a257842291b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:10.322936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 349/722] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:10 2022] GET /identity/v3/role_assignments?user.id=19e0fbe0b5814ad1ae95c65a64b632d6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=03b724e03f704f2ab338b64025a3db6d => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:10.324068 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a18d-6560-48e1-8d93-6a257842291b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:10.339251 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:10.365567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313d857-5100-47b4-9271-a5ed5b2493cd None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:10.426971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309fcd7-c605-46f6-8214-5f06ff592c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:10.428518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309fcd7-c605-46f6-8214-5f06ff592c43 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:10.429066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309fcd7-c605-46f6-8214-5f06ff592c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:10.429574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309fcd7-c605-46f6-8214-5f06ff592c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:10.445909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309fcd7-c605-46f6-8214-5f06ff592c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:10.448270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309fcd7-c605-46f6-8214-5f06ff592c43 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:10.462278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a18d-6560-48e1-8d93-6a257842291b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3309fcd7-c605-46f6-8214-5f06ff592c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:10.471191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:10.472201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:10.473110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:10.473110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:10.473110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:10.473110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:10.475686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 374/723] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:10.477658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1adf55-8262-4d2a-afeb-3a23df566d3b None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:10.478299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1adf55-8262-4d2a-afeb-3a23df566d3b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:10.478890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1adf55-8262-4d2a-afeb-3a23df566d3b None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:10.480941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 375/724] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:10.482071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1adf55-8262-4d2a-afeb-3a23df566d3b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:10.506223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95550ff1-0f50-4bcf-9e53-10aa8a378de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:10.506908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95550ff1-0f50-4bcf-9e53-10aa8a378de9 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:10.507188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95550ff1-0f50-4bcf-9e53-10aa8a378de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:10.507462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95550ff1-0f50-4bcf-9e53-10aa8a378de9 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:10.509042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95550ff1-0f50-4bcf-9e53-10aa8a378de9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:10.509364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95550ff1-0f50-4bcf-9e53-10aa8a378de9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:10.511153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95550ff1-0f50-4bcf-9e53-10aa8a378de9 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:10.521553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 350/725] 10.222.0.64 () {56 vars in 1124 bytes} [Mon Jun 27 09:16:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:10.523698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321cb75-4f7c-4a7c-9128-b17e67567a46 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:10.524092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321cb75-4f7c-4a7c-9128-b17e67567a46 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:10.524462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321cb75-4f7c-4a7c-9128-b17e67567a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:10.533237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:10.548079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d321cb75-4f7c-4a7c-9128-b17e67567a46 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:10.549751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d321cb75-4f7c-4a7c-9128-b17e67567a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:10.624900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1adf55-8262-4d2a-afeb-3a23df566d3b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:10.654241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 351/726] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:10.679257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e184e67-3ffb-40b6-94e4-715c8a13bd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:10.680028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e184e67-3ffb-40b6-94e4-715c8a13bd1f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:10.680312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e184e67-3ffb-40b6-94e4-715c8a13bd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:10.680650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e184e67-3ffb-40b6-94e4-715c8a13bd1f None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:10.681389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e184e67-3ffb-40b6-94e4-715c8a13bd1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:10.681718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e184e67-3ffb-40b6-94e4-715c8a13bd1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:10.683119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e184e67-3ffb-40b6-94e4-715c8a13bd1f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:10.686557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:10.687681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321cb75-4f7c-4a7c-9128-b17e67567a46 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:10.723742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 376/727] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:10.728646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e184e67-3ffb-40b6-94e4-715c8a13bd1f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:10.838191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321cb75-4f7c-4a7c-9128-b17e67567a46 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:10.910287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ca2e4-4a09-4e82-8d0c-4490cc3b4c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:10.911489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ca2e4-4a09-4e82-8d0c-4490cc3b4c81 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:10.912000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ca2e4-4a09-4e82-8d0c-4490cc3b4c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:10.912480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ca2e4-4a09-4e82-8d0c-4490cc3b4c81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:10.916003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:10.933944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ca2e4-4a09-4e82-8d0c-4490cc3b4c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-5a4ca2e4-4a09-4e82-8d0c-4490cc3b4c81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:10.951099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:10.952942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:10.955050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 352/728] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:10.961378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ca2e4-4a09-4e82-8d0c-4490cc3b4c81 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:10.973340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e184e67-3ffb-40b6-94e4-715c8a13bd1f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:11.065247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5f96e-9263-4883-90fb-5a713067872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:11.066083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5f96e-9263-4883-90fb-5a713067872f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:11.066410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5f96e-9263-4883-90fb-5a713067872f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:11.066691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5f96e-9263-4883-90fb-5a713067872f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:11.077817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5f96e-9263-4883-90fb-5a713067872f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:11.079287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5f96e-9263-4883-90fb-5a713067872f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-23c5f96e-9263-4883-90fb-5a713067872f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:11.098972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:11.101285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 377/729] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:16:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd84f2-dba4-4067-b14b-c9f1b7209ed3 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:11.103516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd84f2-dba4-4067-b14b-c9f1b7209ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:11.105254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd84f2-dba4-4067-b14b-c9f1b7209ed3 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:11.108929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 378/730] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:11.109865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd84f2-dba4-4067-b14b-c9f1b7209ed3 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:11.176253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ca2e4-4a09-4e82-8d0c-4490cc3b4c81 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:11.199268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:11.227102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9e3a9-efd0-44bc-a2f7-0db2fe753056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:11.228180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9e3a9-efd0-44bc-a2f7-0db2fe753056 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:11.228648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9e3a9-efd0-44bc-a2f7-0db2fe753056 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:11.229176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9e3a9-efd0-44bc-a2f7-0db2fe753056 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:11.230191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9e3a9-efd0-44bc-a2f7-0db2fe753056 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:11.230666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9e3a9-efd0-44bc-a2f7-0db2fe753056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:11.232319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9e3a9-efd0-44bc-a2f7-0db2fe753056 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:11.244095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 353/731] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:11.245828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37739f-9afd-4ac2-bdf5-58f87daf459b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:11.246285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37739f-9afd-4ac2-bdf5-58f87daf459b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:11.246716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37739f-9afd-4ac2-bdf5-58f87daf459b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:11.266415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-6b37739f-9afd-4ac2-bdf5-58f87daf459b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:11.268685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6b37739f-9afd-4ac2-bdf5-58f87daf459b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:11.277202 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd84f2-dba4-4067-b14b-c9f1b7209ed3 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:11.322103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911c653-155e-4760-bec2-66b038b565e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:11.323227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911c653-155e-4760-bec2-66b038b565e6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:11.323227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911c653-155e-4760-bec2-66b038b565e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:11.323476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911c653-155e-4760-bec2-66b038b565e6 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:11.324289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911c653-155e-4760-bec2-66b038b565e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:11.324607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911c653-155e-4760-bec2-66b038b565e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:11.326595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911c653-155e-4760-bec2-66b038b565e6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:11.337647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:11.348636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 379/732] 10.222.0.64 () {56 vars in 1149 bytes} [Mon Jun 27 09:16:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:11.349646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911c653-155e-4760-bec2-66b038b565e6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:11.360735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 354/733] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:11.362892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37739f-9afd-4ac2-bdf5-58f87daf459b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:11.491909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911c653-155e-4760-bec2-66b038b565e6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:11.508945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:11.532658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26a43e-f3a5-48ab-af97-a888fbc20835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:11.534681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26a43e-f3a5-48ab-af97-a888fbc20835 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:11.534977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26a43e-f3a5-48ab-af97-a888fbc20835 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:11.535260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26a43e-f3a5-48ab-af97-a888fbc20835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:11.548880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26a43e-f3a5-48ab-af97-a888fbc20835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:11.551363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26a43e-f3a5-48ab-af97-a888fbc20835 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-6a26a43e-f3a5-48ab-af97-a888fbc20835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:11.576460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:11.578408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:11.580977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:11.580977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:11.580977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:11.580977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:11.583595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 380/734] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:11.584669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26a43e-f3a5-48ab-af97-a888fbc20835 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:11.589382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37739f-9afd-4ac2-bdf5-58f87daf459b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:11.661979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f4417-f1c9-456f-94bf-2ceba2179ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:11.663810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f4417-f1c9-456f-94bf-2ceba2179ead None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:11.664689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f4417-f1c9-456f-94bf-2ceba2179ead None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:11.665426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f4417-f1c9-456f-94bf-2ceba2179ead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:11.712667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26a43e-f3a5-48ab-af97-a888fbc20835 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:11.758241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcbf7e-e0d4-49bb-a6f5-babb666b6188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:11.758815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcbf7e-e0d4-49bb-a6f5-babb666b6188 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:11.759099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcbf7e-e0d4-49bb-a6f5-babb666b6188 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:11.759366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcbf7e-e0d4-49bb-a6f5-babb666b6188 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:11.760160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcbf7e-e0d4-49bb-a6f5-babb666b6188 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:11.761904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcbf7e-e0d4-49bb-a6f5-babb666b6188 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:11.782764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:11.808977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f4417-f1c9-456f-94bf-2ceba2179ead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:11.811895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f4417-f1c9-456f-94bf-2ceba2179ead None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:11.880004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 381/735] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:16:11 2022] POST /identity/v3/services => generated 240 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:16:11.882608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcbf7e-e0d4-49bb-a6f5-babb666b6188 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:11.952311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:12.119779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 355/736] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 758 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 09:16:12.127705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f4417-f1c9-456f-94bf-2ceba2179ead None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:12.139653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcbf7e-e0d4-49bb-a6f5-babb666b6188 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:12.212590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55741f68-d4a6-4333-9679-d716881af02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:12.213774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55741f68-d4a6-4333-9679-d716881af02a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:12.214281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55741f68-d4a6-4333-9679-d716881af02a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:12.214775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55741f68-d4a6-4333-9679-d716881af02a None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:12.217146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55741f68-d4a6-4333-9679-d716881af02a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:12.217713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55741f68-d4a6-4333-9679-d716881af02a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:12.221811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55741f68-d4a6-4333-9679-d716881af02a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:12.250958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 382/737] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:12.263667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55741f68-d4a6-4333-9679-d716881af02a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:12.368315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:12.399383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f4417-f1c9-456f-94bf-2ceba2179ead None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:12.505748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4204d-096e-4daf-938c-2ace48ca6729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:12.507379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4204d-096e-4daf-938c-2ace48ca6729 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:12.508239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4204d-096e-4daf-938c-2ace48ca6729 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:12.508859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4204d-096e-4daf-938c-2ace48ca6729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:12.537889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:12.553190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4204d-096e-4daf-938c-2ace48ca6729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:12.562181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4204d-096e-4daf-938c-2ace48ca6729 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:12.592840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55741f68-d4a6-4333-9679-d716881af02a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-dae4204d-096e-4daf-938c-2ace48ca6729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:12.606280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:12.607769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:12.608816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:12.608816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:12.608816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:12.608816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:12.609069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 356/738] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 482 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:12.621332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4204d-096e-4daf-938c-2ace48ca6729 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:12.669902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822d8ec-9923-4df9-b90e-5dc35e4b1f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:12.670899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822d8ec-9923-4df9-b90e-5dc35e4b1f1c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:12.671285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822d8ec-9923-4df9-b90e-5dc35e4b1f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:12.671693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822d8ec-9923-4df9-b90e-5dc35e4b1f1c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:12.683517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822d8ec-9923-4df9-b90e-5dc35e4b1f1c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:12.685210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822d8ec-9923-4df9-b90e-5dc35e4b1f1c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-e822d8ec-9923-4df9-b90e-5dc35e4b1f1c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:12.712067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:12.713259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:16:12.714404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:12.718408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 383/739] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:12.724034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822d8ec-9923-4df9-b90e-5dc35e4b1f1c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:12.894426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4204d-096e-4daf-938c-2ace48ca6729 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:12.926250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822d8ec-9923-4df9-b90e-5dc35e4b1f1c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:12.968297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4fcff-2f6a-4fc0-befb-c2104bf22cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:12.969147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4fcff-2f6a-4fc0-befb-c2104bf22cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:12.969529 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4fcff-2f6a-4fc0-befb-c2104bf22cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:12.970021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4fcff-2f6a-4fc0-befb-c2104bf22cd9 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:12.971249 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4fcff-2f6a-4fc0-befb-c2104bf22cd9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:12.971715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4fcff-2f6a-4fc0-befb-c2104bf22cd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:12.973402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4fcff-2f6a-4fc0-befb-c2104bf22cd9 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:12.976697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab952cfb-adc5-4df9-a0b6-df88bde2c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:12.977619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab952cfb-adc5-4df9-a0b6-df88bde2c50d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:12.978087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab952cfb-adc5-4df9-a0b6-df88bde2c50d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:12.978544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab952cfb-adc5-4df9-a0b6-df88bde2c50d None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:12.979480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab952cfb-adc5-4df9-a0b6-df88bde2c50d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:12.979996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab952cfb-adc5-4df9-a0b6-df88bde2c50d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:12.981598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab952cfb-adc5-4df9-a0b6-df88bde2c50d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:12.992074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:12.996340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 384/740] 10.222.0.64 () {56 vars in 1135 bytes} [Mon Jun 27 09:16:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:12.999895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:13.009271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4fcff-2f6a-4fc0-befb-c2104bf22cd9 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:13.013328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 357/741] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:13.020674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab952cfb-adc5-4df9-a0b6-df88bde2c50d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:13.238719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4fcff-2f6a-4fc0-befb-c2104bf22cd9 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:13.244635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab952cfb-adc5-4df9-a0b6-df88bde2c50d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:13.310085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97cb16-e1b4-4686-bc59-5cf645341391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:13.310820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97cb16-e1b4-4686-bc59-5cf645341391 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:13.311114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97cb16-e1b4-4686-bc59-5cf645341391 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:13.311393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97cb16-e1b4-4686-bc59-5cf645341391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:13.340811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d67c5-823b-471d-b3b8-059565643260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:13.341638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d67c5-823b-471d-b3b8-059565643260 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:13.341923 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d67c5-823b-471d-b3b8-059565643260 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:13.342204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d67c5-823b-471d-b3b8-059565643260 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:13.352881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d67c5-823b-471d-b3b8-059565643260 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:13.354394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d67c5-823b-471d-b3b8-059565643260 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-c00d67c5-823b-471d-b3b8-059565643260 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:13.381589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:13.382554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:13.383469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:13.383469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:13.383469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:13.383469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:13.383469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:13.383469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:13.383469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:13.383469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:13.394534 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 358/742] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:13.401864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d67c5-823b-471d-b3b8-059565643260 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:13.464333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:13.468139 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97cb16-e1b4-4686-bc59-5cf645341391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:13.470280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97cb16-e1b4-4686-bc59-5cf645341391 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:13.514681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:13.604060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d67c5-823b-471d-b3b8-059565643260 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:13.614138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 385/743] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:13 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 09:16:13.615794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df206d70-604d-4e93-a259-9378f49157d5 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:13.616181 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df206d70-604d-4e93-a259-9378f49157d5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:13.616540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df206d70-604d-4e93-a259-9378f49157d5 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:13.618379 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 386/744] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:13.622450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df206d70-604d-4e93-a259-9378f49157d5 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:13.667739 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b223b-2869-4078-a801-af7cb4a267c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:13.668466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b223b-2869-4078-a801-af7cb4a267c5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:13.668755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b223b-2869-4078-a801-af7cb4a267c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:13.669040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b223b-2869-4078-a801-af7cb4a267c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:13.670322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b223b-2869-4078-a801-af7cb4a267c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:13.670638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b223b-2869-4078-a801-af7cb4a267c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:13.672402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b223b-2869-4078-a801-af7cb4a267c5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:13.689141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 359/745] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:13.691155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce492f-2797-4dd5-b535-ba2a1b5ecc33 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:13.691480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce492f-2797-4dd5-b535-ba2a1b5ecc33 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:13.691773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce492f-2797-4dd5-b535-ba2a1b5ecc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:13.712734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ebce492f-2797-4dd5-b535-ba2a1b5ecc33 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:13.713962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ebce492f-2797-4dd5-b535-ba2a1b5ecc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:13.790643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:13.792925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df206d70-604d-4e93-a259-9378f49157d5 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:13.800900 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 360/746] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:13 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 09:16:13.808451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce492f-2797-4dd5-b535-ba2a1b5ecc33 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:13.837166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a29e1e-e0b7-4167-b31a-0d9760967d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:13.838016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a29e1e-e0b7-4167-b31a-0d9760967d1a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:13.838296 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a29e1e-e0b7-4167-b31a-0d9760967d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:13.838571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a29e1e-e0b7-4167-b31a-0d9760967d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:13.853991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a29e1e-e0b7-4167-b31a-0d9760967d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:13.856197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a29e1e-e0b7-4167-b31a-0d9760967d1a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:13.881713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-44a29e1e-e0b7-4167-b31a-0d9760967d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:13.884683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:13.885814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:13.886934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:13.886934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:13.886934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:13.886934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:13.886934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 387/747] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:13.887307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a29e1e-e0b7-4167-b31a-0d9760967d1a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:13.944211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce492f-2797-4dd5-b535-ba2a1b5ecc33 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:13.984772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ae96e-cc32-47a3-8dbf-6fb197b82b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:13.985603 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ae96e-cc32-47a3-8dbf-6fb197b82b63 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:13.986003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ae96e-cc32-47a3-8dbf-6fb197b82b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:13.986377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ae96e-cc32-47a3-8dbf-6fb197b82b63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:13.987922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ae96e-cc32-47a3-8dbf-6fb197b82b63 None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=8731ac7e5a7941f1b2ad04bddc966db6) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:13.988435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ae96e-cc32-47a3-8dbf-6fb197b82b63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:13.990094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ae96e-cc32-47a3-8dbf-6fb197b82b63 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:13.997455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 361/748] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:13 2022] GET /identity/v3/role_assignments?user.id=8731ac7e5a7941f1b2ad04bddc966db6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:13.998574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ae96e-cc32-47a3-8dbf-6fb197b82b63 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:14.018740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a29e1e-e0b7-4167-b31a-0d9760967d1a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:14.063215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee420de-2927-4f53-81ec-9ae42da6b751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:14.063899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee420de-2927-4f53-81ec-9ae42da6b751 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:14.064289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee420de-2927-4f53-81ec-9ae42da6b751 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:14.064606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee420de-2927-4f53-81ec-9ae42da6b751 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:14.066022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee420de-2927-4f53-81ec-9ae42da6b751 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:14.067576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee420de-2927-4f53-81ec-9ae42da6b751 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:14.126077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ae96e-cc32-47a3-8dbf-6fb197b82b63 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:14.145034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:14.173964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf43281-9c26-41e0-a2c7-7ad0b02a9a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:14.175372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf43281-9c26-41e0-a2c7-7ad0b02a9a6e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:14.175836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf43281-9c26-41e0-a2c7-7ad0b02a9a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:14.176243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf43281-9c26-41e0-a2c7-7ad0b02a9a6e None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:14.177151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf43281-9c26-41e0-a2c7-7ad0b02a9a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:14.177593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf43281-9c26-41e0-a2c7-7ad0b02a9a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:14.179820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf43281-9c26-41e0-a2c7-7ad0b02a9a6e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:14.200523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 388/749] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:16:13 2022] POST /identity/v3/services => generated 251 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:16:14.200523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e56ab5-8bf4-4852-be66-0c267d5f1f03 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:14.200523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e56ab5-8bf4-4852-be66-0c267d5f1f03 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:14.200523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e56ab5-8bf4-4852-be66-0c267d5f1f03 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:14.208257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 389/750] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:14.212056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 362/751] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:14.217269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e816dd-0f0b-4dcb-beed-9e0abb67cc95 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:14.217739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e816dd-0f0b-4dcb-beed-9e0abb67cc95 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:14.218209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e816dd-0f0b-4dcb-beed-9e0abb67cc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:14.223888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf43281-9c26-41e0-a2c7-7ad0b02a9a6e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:14.317009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-69e816dd-0f0b-4dcb-beed-9e0abb67cc95 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:14.318147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-69e816dd-0f0b-4dcb-beed-9e0abb67cc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:14.332603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:14.481980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 390/752] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:14.491097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196a63f-d642-44f1-9f08-98592e010200 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:14.491097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196a63f-d642-44f1-9f08-98592e010200 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:14.491380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196a63f-d642-44f1-9f08-98592e010200 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:14.493247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 391/753] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:14.498548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159999a-ab46-40d4-a417-cfa00ac2c3eb None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:14.498879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159999a-ab46-40d4-a417-cfa00ac2c3eb None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:14.499125 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159999a-ab46-40d4-a417-cfa00ac2c3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:14.509798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf43281-9c26-41e0-a2c7-7ad0b02a9a6e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:14.519394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-9159999a-ab46-40d4-a417-cfa00ac2c3eb None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:14.520804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9159999a-ab46-40d4-a417-cfa00ac2c3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:14.561989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcd8a0-452a-4e39-b5c0-121deec879eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:14.562813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcd8a0-452a-4e39-b5c0-121deec879eb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:14.563095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcd8a0-452a-4e39-b5c0-121deec879eb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:14.563367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcd8a0-452a-4e39-b5c0-121deec879eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:14.574627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcd8a0-452a-4e39-b5c0-121deec879eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:14.576062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcd8a0-452a-4e39-b5c0-121deec879eb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-04dcd8a0-452a-4e39-b5c0-121deec879eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:14.593084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:14.594094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:14.594988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:14.594988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:14.594988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:14.594988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:14.594988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:14.594988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:14.594988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:14.594988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:14.594988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 363/754] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:14.595494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcd8a0-452a-4e39-b5c0-121deec879eb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:14.629131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 392/755] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:14.630994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159999a-ab46-40d4-a417-cfa00ac2c3eb None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:14.647680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:14.719970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:14.753860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcd8a0-452a-4e39-b5c0-121deec879eb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:14.756457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159999a-ab46-40d4-a417-cfa00ac2c3eb None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:14.799976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f19f96-fe29-48fa-b340-c5c8a4c6f9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:14.801234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f19f96-fe29-48fa-b340-c5c8a4c6f9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:14.801686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f19f96-fe29-48fa-b340-c5c8a4c6f9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:14.802208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f19f96-fe29-48fa-b340-c5c8a4c6f9a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:14.803255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018fbd0-6ccc-40d9-b77b-21c39e8f451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:14.804667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018fbd0-6ccc-40d9-b77b-21c39e8f451a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:14.805143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018fbd0-6ccc-40d9-b77b-21c39e8f451a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:14.805763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018fbd0-6ccc-40d9-b77b-21c39e8f451a None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:14.807439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018fbd0-6ccc-40d9-b77b-21c39e8f451a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:14.808276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018fbd0-6ccc-40d9-b77b-21c39e8f451a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:14.811118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018fbd0-6ccc-40d9-b77b-21c39e8f451a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:14.816258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f19f96-fe29-48fa-b340-c5c8a4c6f9a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-d8f19f96-fe29-48fa-b340-c5c8a4c6f9a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:14.825229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:14.826222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:14.827087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:14.836216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 364/756] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:14.837348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f19f96-fe29-48fa-b340-c5c8a4c6f9a3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:14.840347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 393/757] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:14.841509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018fbd0-6ccc-40d9-b77b-21c39e8f451a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:14.981648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018fbd0-6ccc-40d9-b77b-21c39e8f451a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:14.984281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:14.997521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f19f96-fe29-48fa-b340-c5c8a4c6f9a3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:15.024104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:15.032244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1faa60c-eb87-401a-8212-a06ee142b70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:15.033370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1faa60c-eb87-401a-8212-a06ee142b70f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:15.033883 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1faa60c-eb87-401a-8212-a06ee142b70f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:15.034551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1faa60c-eb87-401a-8212-a06ee142b70f None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:15.035998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1faa60c-eb87-401a-8212-a06ee142b70f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:15.036856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1faa60c-eb87-401a-8212-a06ee142b70f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:15.038765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1faa60c-eb87-401a-8212-a06ee142b70f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:15.045395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cab84-ce07-4baa-b8ab-aa84c55aec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:15.046325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cab84-ce07-4baa-b8ab-aa84c55aec10 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:15.047015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cab84-ce07-4baa-b8ab-aa84c55aec10 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:15.047015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cab84-ce07-4baa-b8ab-aa84c55aec10 None admin] PATH_INFO: `/v3/services/network` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:15.047850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cab84-ce07-4baa-b8ab-aa84c55aec10 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:15.049978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cab84-ce07-4baa-b8ab-aa84c55aec10 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:15.056613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 394/758] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:16:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:15.057759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1faa60c-eb87-401a-8212-a06ee142b70f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0f9cab84-ce07-4baa-b8ab-aa84c55aec10 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:15.081752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:15.082822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 09:16:15.083700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:15.091960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 365/759] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:14 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:15.094003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cab84-ce07-4baa-b8ab-aa84c55aec10 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:15.195876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1faa60c-eb87-401a-8212-a06ee142b70f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:15.241766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbb18f-b9d2-4673-a838-02cd5f7fc463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:15.243033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbb18f-b9d2-4673-a838-02cd5f7fc463 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:15.243555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbb18f-b9d2-4673-a838-02cd5f7fc463 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:15.244465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbb18f-b9d2-4673-a838-02cd5f7fc463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:15.245850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbb18f-b9d2-4673-a838-02cd5f7fc463 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=70b499a32516486d9a2bed34f7e66b76) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:15.246420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbb18f-b9d2-4673-a838-02cd5f7fc463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:15.248404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbb18f-b9d2-4673-a838-02cd5f7fc463 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:15.257500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 395/760] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:15 2022] GET /identity/v3/role_assignments?user.id=70b499a32516486d9a2bed34f7e66b76&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=03b724e03f704f2ab338b64025a3db6d => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:15.258753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbb18f-b9d2-4673-a838-02cd5f7fc463 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:15.259761 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cab84-ce07-4baa-b8ab-aa84c55aec10 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:15.341294 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:15.401091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbb18f-b9d2-4673-a838-02cd5f7fc463 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:15.424872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6743dc-79ba-4ece-93b5-ffe780189347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:15.425700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6743dc-79ba-4ece-93b5-ffe780189347 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:15.425984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6743dc-79ba-4ece-93b5-ffe780189347 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:15.426261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6743dc-79ba-4ece-93b5-ffe780189347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:15.445167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972f836-d8a2-4ac0-84d4-ae766bf2f927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:15.445656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f836-d8a2-4ac0-84d4-ae766bf2f927 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:15.445944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f836-d8a2-4ac0-84d4-ae766bf2f927 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:15.446223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972f836-d8a2-4ac0-84d4-ae766bf2f927 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:15.447703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972f836-d8a2-4ac0-84d4-ae766bf2f927 None admin] RBAC: Adding query filter params (name=network) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:15.447703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972f836-d8a2-4ac0-84d4-ae766bf2f927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:15.449284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972f836-d8a2-4ac0-84d4-ae766bf2f927 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:15.455398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 396/761] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:15.458156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6743dc-79ba-4ece-93b5-ffe780189347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:15.460832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6743dc-79ba-4ece-93b5-ffe780189347 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:15.462889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972f836-d8a2-4ac0-84d4-ae766bf2f927 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-9c6743dc-79ba-4ece-93b5-ffe780189347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:15.480451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:15.481555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:15.482843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:15.482843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:15.482843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:15.482843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:15.482843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 366/762] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:15.489998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6743dc-79ba-4ece-93b5-ffe780189347 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:15.515253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:15.684749 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972f836-d8a2-4ac0-84d4-ae766bf2f927 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:15.716303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6743dc-79ba-4ece-93b5-ffe780189347 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:15.731892 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:15.759671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989dac1-629c-422c-bcb3-9939df505163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:15.760348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989dac1-629c-422c-bcb3-9939df505163 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:15.760630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989dac1-629c-422c-bcb3-9939df505163 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:15.760912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989dac1-629c-422c-bcb3-9939df505163 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:15.761746 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989dac1-629c-422c-bcb3-9939df505163 None admin] RBAC: Adding query filter params (type=network) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:15.762056 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989dac1-629c-422c-bcb3-9939df505163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:15.763433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989dac1-629c-422c-bcb3-9939df505163 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:15.776138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 397/763] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:15.780649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78459a4-6cb7-4667-a913-6e2bb40ac088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:15.781342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78459a4-6cb7-4667-a913-6e2bb40ac088 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:15.781627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78459a4-6cb7-4667-a913-6e2bb40ac088 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:15.781911 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78459a4-6cb7-4667-a913-6e2bb40ac088 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:15.782650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78459a4-6cb7-4667-a913-6e2bb40ac088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:15.782963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78459a4-6cb7-4667-a913-6e2bb40ac088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:15.784983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78459a4-6cb7-4667-a913-6e2bb40ac088 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:15.787303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989dac1-629c-422c-bcb3-9939df505163 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:15.813726 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 367/764] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:15.818866 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78459a4-6cb7-4667-a913-6e2bb40ac088 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:15.949688 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989dac1-629c-422c-bcb3-9939df505163 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:15.959940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:16.006338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78459a4-6cb7-4667-a913-6e2bb40ac088 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:16.021752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981035a-0cac-4227-b54b-e90934134590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:16.022432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981035a-0cac-4227-b54b-e90934134590 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.022708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981035a-0cac-4227-b54b-e90934134590 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.022980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981035a-0cac-4227-b54b-e90934134590 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.023794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981035a-0cac-4227-b54b-e90934134590 None admin] RBAC: Adding query filter params (interface=public, service_id=43ae88369ecb41c9a41dc103e4665cb1, region_id=RegionOne) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:16.024158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981035a-0cac-4227-b54b-e90934134590 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:16.025806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981035a-0cac-4227-b54b-e90934134590 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:16.042028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 398/765] 10.222.0.64 () {56 vars in 1251 bytes} [Mon Jun 27 09:16:15 2022] GET /identity/v3/endpoints?service_id=43ae88369ecb41c9a41dc103e4665cb1&interface=public®ion_id=RegionOne => generated 194 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:16.046532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981035a-0cac-4227-b54b-e90934134590 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:16.074575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eadc2-1584-48ac-97ae-01669be70edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:16.075387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eadc2-1584-48ac-97ae-01669be70edb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.075736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eadc2-1584-48ac-97ae-01669be70edb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.076025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eadc2-1584-48ac-97ae-01669be70edb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.084810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9eadc2-1584-48ac-97ae-01669be70edb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:16.086289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9eadc2-1584-48ac-97ae-01669be70edb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-8d9eadc2-1584-48ac-97ae-01669be70edb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:16.104582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:16.105592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:16.107000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 368/766] 10.222.0.64 () {56 vars in 1093 bytes} [Mon Jun 27 09:16:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:16.115024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eadc2-1584-48ac-97ae-01669be70edb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:16.155910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:16.266027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eadc2-1584-48ac-97ae-01669be70edb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:16.288877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981035a-0cac-4227-b54b-e90934134590 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:16.319938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:16.330038 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a43cd-5ad1-449c-aa0b-bdcfb7843176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:16.333406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a43cd-5ad1-449c-aa0b-bdcfb7843176 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.333406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a43cd-5ad1-449c-aa0b-bdcfb7843176 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.333406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a43cd-5ad1-449c-aa0b-bdcfb7843176 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.333406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a43cd-5ad1-449c-aa0b-bdcfb7843176 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:16.333406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a43cd-5ad1-449c-aa0b-bdcfb7843176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:16.334541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a43cd-5ad1-449c-aa0b-bdcfb7843176 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:16.367224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433f692-ada5-4792-a1b3-c322fc7d5bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:16.368000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433f692-ada5-4792-a1b3-c322fc7d5bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.368207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433f692-ada5-4792-a1b3-c322fc7d5bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.368482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433f692-ada5-4792-a1b3-c322fc7d5bc2 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.369172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433f692-ada5-4792-a1b3-c322fc7d5bc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:16.370574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433f692-ada5-4792-a1b3-c322fc7d5bc2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:16.375077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 369/767] 10.222.0.64 () {56 vars in 1133 bytes} [Mon Jun 27 09:16:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:16.381167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a43cd-5ad1-449c-aa0b-bdcfb7843176 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:16.390877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 399/768] 10.222.0.64 () {56 vars in 1087 bytes} [Mon Jun 27 09:16:16 2022] GET /identity/v3/services => generated 772 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:16.463447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e10a4-6c3a-4856-9d24-98f44b1fe51c None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.464365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e10a4-6c3a-4856-9d24-98f44b1fe51c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.465158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e10a4-6c3a-4856-9d24-98f44b1fe51c None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.468484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 400/769] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:16.478007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef2413-7d60-448d-b48a-88471afe1393 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.478991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef2413-7d60-448d-b48a-88471afe1393 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.479782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef2413-7d60-448d-b48a-88471afe1393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.526419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e1ef2413-7d60-448d-b48a-88471afe1393 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:16.529099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e1ef2413-7d60-448d-b48a-88471afe1393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:16.581082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a43cd-5ad1-449c-aa0b-bdcfb7843176 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:16.620840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 401/770] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:16.628501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7aba2-30e4-4a47-902e-0cc81ff8baef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:16.629322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7aba2-30e4-4a47-902e-0cc81ff8baef None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.629607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7aba2-30e4-4a47-902e-0cc81ff8baef None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.629881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7aba2-30e4-4a47-902e-0cc81ff8baef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.641470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7aba2-30e4-4a47-902e-0cc81ff8baef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:16.643507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7aba2-30e4-4a47-902e-0cc81ff8baef None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:16.647697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef2413-7d60-448d-b48a-88471afe1393 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-7bd7aba2-30e4-4a47-902e-0cc81ff8baef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:16.667899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:16.669039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:16.670194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:16.670194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:16.670194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:16.670194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:16.670194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 370/771] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:16.675988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7aba2-30e4-4a47-902e-0cc81ff8baef None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:16.719752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:16.755281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:16.808813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef2413-7d60-448d-b48a-88471afe1393 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:16.859149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fc008-36db-4e24-9750-f57fb1e2e9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:16.860573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fc008-36db-4e24-9750-f57fb1e2e9ea None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.860687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7aba2-30e4-4a47-902e-0cc81ff8baef None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:16.861111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fc008-36db-4e24-9750-f57fb1e2e9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.862109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fc008-36db-4e24-9750-f57fb1e2e9ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.884152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fc008-36db-4e24-9750-f57fb1e2e9ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-c68fc008-36db-4e24-9750-f57fb1e2e9ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:16.892774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:16.893733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:16.894564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:16.899175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 402/772] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:16:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:16.901253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa9c21-95ce-41f1-8073-efa2398e7ecb None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.901641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa9c21-95ce-41f1-8073-efa2398e7ecb None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.901930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa9c21-95ce-41f1-8073-efa2398e7ecb None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.903615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 403/773] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:16.904495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa9c21-95ce-41f1-8073-efa2398e7ecb None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:16.924830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934b5c7-7672-4ec6-a8ed-eef9e6fe504f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:16.925518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b5c7-7672-4ec6-a8ed-eef9e6fe504f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.925851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b5c7-7672-4ec6-a8ed-eef9e6fe504f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.926134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b5c7-7672-4ec6-a8ed-eef9e6fe504f None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.927058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934b5c7-7672-4ec6-a8ed-eef9e6fe504f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:16.927627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934b5c7-7672-4ec6-a8ed-eef9e6fe504f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:16.929810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934b5c7-7672-4ec6-a8ed-eef9e6fe504f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:16.948120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 371/774] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:16.949862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5107f3-03ad-42bd-aa91-513f725d122a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:16.950186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5107f3-03ad-42bd-aa91-513f725d122a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:16.950554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5107f3-03ad-42bd-aa91-513f725d122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:16.973350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7d5107f3-03ad-42bd-aa91-513f725d122a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:16.974544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7d5107f3-03ad-42bd-aa91-513f725d122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:17.053263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 372/775] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:17.054224 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5107f3-03ad-42bd-aa91-513f725d122a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:17.084879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:17.086575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:17.091841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa9c21-95ce-41f1-8073-efa2398e7ecb None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:17.130581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c846e-3c4a-4856-98e1-827d30034192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:17.131275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c846e-3c4a-4856-98e1-827d30034192 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:17.131594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c846e-3c4a-4856-98e1-827d30034192 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:17.131873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c846e-3c4a-4856-98e1-827d30034192 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:17.133380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c846e-3c4a-4856-98e1-827d30034192 None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:17.134140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c846e-3c4a-4856-98e1-827d30034192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:17.136211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c846e-3c4a-4856-98e1-827d30034192 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:17.153670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 404/776] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:16:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:17.153670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c846e-3c4a-4856-98e1-827d30034192 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:17.193913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5107f3-03ad-42bd-aa91-513f725d122a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:17.242663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bafac-3068-46e5-b417-a9829f4974ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:17.243613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bafac-3068-46e5-b417-a9829f4974ba None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:17.243970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bafac-3068-46e5-b417-a9829f4974ba None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:17.244321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bafac-3068-46e5-b417-a9829f4974ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:17.254393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bafac-3068-46e5-b417-a9829f4974ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:17.256015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bafac-3068-46e5-b417-a9829f4974ba None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-b88bafac-3068-46e5-b417-a9829f4974ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:17.274778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:17.277047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 373/777] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:17.279967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bafac-3068-46e5-b417-a9829f4974ba None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:17.320110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c846e-3c4a-4856-98e1-827d30034192 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:17.380067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de1b7-3b5a-411e-bcb0-c31cf1526363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:17.382588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de1b7-3b5a-411e-bcb0-c31cf1526363 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:17.383364 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de1b7-3b5a-411e-bcb0-c31cf1526363 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:17.384159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de1b7-3b5a-411e-bcb0-c31cf1526363 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:17.385716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de1b7-3b5a-411e-bcb0-c31cf1526363 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:17.388615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de1b7-3b5a-411e-bcb0-c31cf1526363 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:17.398863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-671de1b7-3b5a-411e-bcb0-c31cf1526363 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:17.412823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:17.418961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 09:16:17.420958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:17.422076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bafac-3068-46e5-b417-a9829f4974ba None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:17.422212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 405/778] 10.222.0.64 () {56 vars in 1129 bytes} [Mon Jun 27 09:16:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:17.422212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de1b7-3b5a-411e-bcb0-c31cf1526363 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:17.456575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:17.467057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341517c-174d-4b98-8c17-6579024dda1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:17.471683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341517c-174d-4b98-8c17-6579024dda1a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:17.471683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341517c-174d-4b98-8c17-6579024dda1a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:17.471683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341517c-174d-4b98-8c17-6579024dda1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:17.481349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341517c-174d-4b98-8c17-6579024dda1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:17.483380 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341517c-174d-4b98-8c17-6579024dda1a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-5341517c-174d-4b98-8c17-6579024dda1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:17.508055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:17.509009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:17.509902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:17.509902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:17.509902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:17.509902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:17.512374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 374/779] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:17.513540 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341517c-174d-4b98-8c17-6579024dda1a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:17.579163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de1b7-3b5a-411e-bcb0-c31cf1526363 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:17.626258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e77f9f-31b3-451a-842e-67ebb66d7643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:17.627722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e77f9f-31b3-451a-842e-67ebb66d7643 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:17.628084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e77f9f-31b3-451a-842e-67ebb66d7643 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:17.628431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e77f9f-31b3-451a-842e-67ebb66d7643 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:17.629313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e77f9f-31b3-451a-842e-67ebb66d7643 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:17.629724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e77f9f-31b3-451a-842e-67ebb66d7643 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:17.631335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e77f9f-31b3-451a-842e-67ebb66d7643 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:17.650095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 406/780] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:17.650095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e77f9f-31b3-451a-842e-67ebb66d7643 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:17.670070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341517c-174d-4b98-8c17-6579024dda1a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:17.715107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d633a-3da9-4c29-8faa-0d866940d225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:17.715883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d633a-3da9-4c29-8faa-0d866940d225 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:17.716167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d633a-3da9-4c29-8faa-0d866940d225 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:17.716442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d633a-3da9-4c29-8faa-0d866940d225 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:17.717234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d633a-3da9-4c29-8faa-0d866940d225 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:17.717552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d633a-3da9-4c29-8faa-0d866940d225 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:17.719161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d633a-3da9-4c29-8faa-0d866940d225 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:17.729568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 375/781] 10.222.0.64 () {56 vars in 1124 bytes} [Mon Jun 27 09:16:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:17.729568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d633a-3da9-4c29-8faa-0d866940d225 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:17.734767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:17.768170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:17.789283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e77f9f-31b3-451a-842e-67ebb66d7643 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:17.839574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f797ae-36b8-41e0-a47f-4228a9134548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:17.840234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f797ae-36b8-41e0-a47f-4228a9134548 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:17.840526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f797ae-36b8-41e0-a47f-4228a9134548 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:17.840778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f797ae-36b8-41e0-a47f-4228a9134548 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:17.841833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f797ae-36b8-41e0-a47f-4228a9134548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:17.842143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f797ae-36b8-41e0-a47f-4228a9134548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:17.843559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f797ae-36b8-41e0-a47f-4228a9134548 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:17.862865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 407/782] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:17.862865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f797ae-36b8-41e0-a47f-4228a9134548 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:17.902901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d633a-3da9-4c29-8faa-0d866940d225 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:17.944817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec61268-aa8d-4ede-88f1-507d5ba33be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:17.946738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec61268-aa8d-4ede-88f1-507d5ba33be8 None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:17.947026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec61268-aa8d-4ede-88f1-507d5ba33be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:17.947871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec61268-aa8d-4ede-88f1-507d5ba33be8 None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19e0fbe0b5814ad1ae95c65a64b632d6/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:17.949875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec61268-aa8d-4ede-88f1-507d5ba33be8 None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=19e0fbe0b5814ad1ae95c65a64b632d6, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:17.952007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec61268-aa8d-4ede-88f1-507d5ba33be8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:17.979096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f797ae-36b8-41e0-a47f-4228a9134548 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:18.020972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 376/783] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:16:17 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19e0fbe0b5814ad1ae95c65a64b632d6/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:16:18.022278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec61268-aa8d-4ede-88f1-507d5ba33be8 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:18.037730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f199dfd-5dd4-4cc2-9451-2f1decf111ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:18.038844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f199dfd-5dd4-4cc2-9451-2f1decf111ce None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.039475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f199dfd-5dd4-4cc2-9451-2f1decf111ce None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.040024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f199dfd-5dd4-4cc2-9451-2f1decf111ce None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.041407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f199dfd-5dd4-4cc2-9451-2f1decf111ce None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:18.042228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f199dfd-5dd4-4cc2-9451-2f1decf111ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:18.045333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f199dfd-5dd4-4cc2-9451-2f1decf111ce None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:18.057042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 408/784] 10.222.0.64 () {56 vars in 1124 bytes} [Mon Jun 27 09:16:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:18.060179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ea4d3-4097-4e7c-8853-5760c11b7975 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.061076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ea4d3-4097-4e7c-8853-5760c11b7975 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.061811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ea4d3-4097-4e7c-8853-5760c11b7975 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.072000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 409/785] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:18 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:18.074416 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ea4d3-4097-4e7c-8853-5760c11b7975 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:18.108534 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:18.122082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:18.174757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec61268-aa8d-4ede-88f1-507d5ba33be8 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:18.218175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20236566-6740-4d8a-bc70-29217b19b367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:18.219056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236566-6740-4d8a-bc70-29217b19b367 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.219385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236566-6740-4d8a-bc70-29217b19b367 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.219783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236566-6740-4d8a-bc70-29217b19b367 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.231598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20236566-6740-4d8a-bc70-29217b19b367 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:18.233099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20236566-6740-4d8a-bc70-29217b19b367 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-20236566-6740-4d8a-bc70-29217b19b367 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:18.250310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:18.251703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:18.252854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 377/786] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:16:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:18.253506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799a318-1795-41f0-8d5e-29fdd188fc8c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.253506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799a318-1795-41f0-8d5e-29fdd188fc8c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.253677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799a318-1795-41f0-8d5e-29fdd188fc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.257567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ea4d3-4097-4e7c-8853-5760c11b7975 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:18.272948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1799a318-1795-41f0-8d5e-29fdd188fc8c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:18.273892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1799a318-1795-41f0-8d5e-29fdd188fc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:18.306108 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6f39-a237-4361-9b1b-85bec6fdef38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:18.306846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6f39-a237-4361-9b1b-85bec6fdef38 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.307132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6f39-a237-4361-9b1b-85bec6fdef38 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.307412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6f39-a237-4361-9b1b-85bec6fdef38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.308244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6f39-a237-4361-9b1b-85bec6fdef38 None admin] RBAC: Adding query filter params (interface=public, service_id=3c4c6b53584747c991287f03ea2eac3b, region_id=RegionOne) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:18.308777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6f39-a237-4361-9b1b-85bec6fdef38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:18.310007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6f39-a237-4361-9b1b-85bec6fdef38 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:18.315696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 410/787] 10.222.0.64 () {56 vars in 1251 bytes} [Mon Jun 27 09:16:18 2022] GET /identity/v3/endpoints?service_id=3c4c6b53584747c991287f03ea2eac3b&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:18.316517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6f39-a237-4361-9b1b-85bec6fdef38 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:18.378005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 378/788] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:18.378871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799a318-1795-41f0-8d5e-29fdd188fc8c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:18.421458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:18.455946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6f39-a237-4361-9b1b-85bec6fdef38 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:18.467055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:18.514756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026631bd-f99f-4768-b8da-238b5446ff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:18.515712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026631bd-f99f-4768-b8da-238b5446ff5b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.516414 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026631bd-f99f-4768-b8da-238b5446ff5b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.516827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026631bd-f99f-4768-b8da-238b5446ff5b None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.518996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026631bd-f99f-4768-b8da-238b5446ff5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:18.519689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026631bd-f99f-4768-b8da-238b5446ff5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:18.521979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026631bd-f99f-4768-b8da-238b5446ff5b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:18.529859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799a318-1795-41f0-8d5e-29fdd188fc8c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:18.553214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 411/789] 10.222.0.64 () {56 vars in 1149 bytes} [Mon Jun 27 09:16:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:18.554348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026631bd-f99f-4768-b8da-238b5446ff5b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:18.600344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ddbb9-95e6-4c44-8715-ec1595897f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:18.601720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ddbb9-95e6-4c44-8715-ec1595897f8f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.602346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ddbb9-95e6-4c44-8715-ec1595897f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.602952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ddbb9-95e6-4c44-8715-ec1595897f8f None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.605376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ddbb9-95e6-4c44-8715-ec1595897f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:18.609543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ddbb9-95e6-4c44-8715-ec1595897f8f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:18.627701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 379/790] 10.222.0.64 () {56 vars in 1087 bytes} [Mon Jun 27 09:16:18 2022] GET /identity/v3/services => generated 772 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:18.628844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ddbb9-95e6-4c44-8715-ec1595897f8f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:18.707017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026631bd-f99f-4768-b8da-238b5446ff5b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:18.757785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:18.764451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ef684-f96a-47b3-88a4-a77cb60f2f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:18.765888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ef684-f96a-47b3-88a4-a77cb60f2f0f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.766191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ef684-f96a-47b3-88a4-a77cb60f2f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.770505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ef684-f96a-47b3-88a4-a77cb60f2f0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.777444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ddbb9-95e6-4c44-8715-ec1595897f8f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:18.782088 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ef684-f96a-47b3-88a4-a77cb60f2f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-0e3ef684-f96a-47b3-88a4-a77cb60f2f0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:18.792979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:18.794176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:18.795158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:18.798686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 412/791] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:18.801201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35625c-a818-4b91-bcb0-31fa1a4820d7 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.802152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35625c-a818-4b91-bcb0-31fa1a4820d7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.802580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35625c-a818-4b91-bcb0-31fa1a4820d7 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.809257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 413/792] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:18.809459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:18.809579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35625c-a818-4b91-bcb0-31fa1a4820d7 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:18.827271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed842b-a3fd-4952-b82e-08b5cd349ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:18.828555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed842b-a3fd-4952-b82e-08b5cd349ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.828910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed842b-a3fd-4952-b82e-08b5cd349ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.829284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed842b-a3fd-4952-b82e-08b5cd349ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.842931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed842b-a3fd-4952-b82e-08b5cd349ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:18.844681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed842b-a3fd-4952-b82e-08b5cd349ad8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-69ed842b-a3fd-4952-b82e-08b5cd349ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:18.869673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:18.871061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:18.872490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:18.872490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:18.872490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:18.872490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:18.873476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 380/793] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:18.874770 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023ffa9-6a78-426d-a3bd-64d66e355974 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:18.875052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023ffa9-6a78-426d-a3bd-64d66e355974 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:18.875321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023ffa9-6a78-426d-a3bd-64d66e355974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:18.896665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8023ffa9-6a78-426d-a3bd-64d66e355974 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:18.897891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8023ffa9-6a78-426d-a3bd-64d66e355974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:18.954838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35625c-a818-4b91-bcb0-31fa1a4820d7 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:18.999667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef27f0-ee59-4ea9-bc56-e03ae929a936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:19.000854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef27f0-ee59-4ea9-bc56-e03ae929a936 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:19.001398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef27f0-ee59-4ea9-bc56-e03ae929a936 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:19.002030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef27f0-ee59-4ea9-bc56-e03ae929a936 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:19.003093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef27f0-ee59-4ea9-bc56-e03ae929a936 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:19.003675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef27f0-ee59-4ea9-bc56-e03ae929a936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:19.005643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef27f0-ee59-4ea9-bc56-e03ae929a936 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:19.011631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 381/794] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:19.015192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023ffa9-6a78-426d-a3bd-64d66e355974 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.036479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 414/795] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:16:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:19.042148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef27f0-ee59-4ea9-bc56-e03ae929a936 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.119733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:19.156368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:19.161258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023ffa9-6a78-426d-a3bd-64d66e355974 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:19.186868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef27f0-ee59-4ea9-bc56-e03ae929a936 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:19.201525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b985c-7bb4-4ccf-9e36-1a14a7b75757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:19.202569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b985c-7bb4-4ccf-9e36-1a14a7b75757 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:19.203178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b985c-7bb4-4ccf-9e36-1a14a7b75757 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:19.203747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b985c-7bb4-4ccf-9e36-1a14a7b75757 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:19.204515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b985c-7bb4-4ccf-9e36-1a14a7b75757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:19.204830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b985c-7bb4-4ccf-9e36-1a14a7b75757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:19.206762 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b985c-7bb4-4ccf-9e36-1a14a7b75757 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:19.227627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 382/796] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:19.229037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b985c-7bb4-4ccf-9e36-1a14a7b75757 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.231560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd108457-c375-4c02-9e15-a3551e033b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:19.232711 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd108457-c375-4c02-9e15-a3551e033b00 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:19.232994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd108457-c375-4c02-9e15-a3551e033b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:19.234281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd108457-c375-4c02-9e15-a3551e033b00 None admin] PATH_INFO: `/v3/services/network` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:19.234975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd108457-c375-4c02-9e15-a3551e033b00 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:19.236679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd108457-c375-4c02-9e15-a3551e033b00 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-bd108457-c375-4c02-9e15-a3551e033b00 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:19.253733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:19.254765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 09:16:19.255652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:19.256401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 415/797] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:19 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:19.258205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd108457-c375-4c02-9e15-a3551e033b00 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.366241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b985c-7bb4-4ccf-9e36-1a14a7b75757 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:19.404495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39990200-1225-4ee1-b8a1-c27d8506a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:19.405589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39990200-1225-4ee1-b8a1-c27d8506a02a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:19.406027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39990200-1225-4ee1-b8a1-c27d8506a02a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:19.406517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39990200-1225-4ee1-b8a1-c27d8506a02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:19.412509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd108457-c375-4c02-9e15-a3551e033b00 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:19.419812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39990200-1225-4ee1-b8a1-c27d8506a02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:19.421529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39990200-1225-4ee1-b8a1-c27d8506a02a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:19.431963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-39990200-1225-4ee1-b8a1-c27d8506a02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:19.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:19.448722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:19.449846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:19.449846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:19.449846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:19.449846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:19.450301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 383/798] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:19.453421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39990200-1225-4ee1-b8a1-c27d8506a02a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.455556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eb431-5962-4cd9-bd97-edff674aafb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:19.457012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eb431-5962-4cd9-bd97-edff674aafb6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:19.457402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eb431-5962-4cd9-bd97-edff674aafb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:19.458019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eb431-5962-4cd9-bd97-edff674aafb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:19.466156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eb431-5962-4cd9-bd97-edff674aafb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:19.468118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eb431-5962-4cd9-bd97-edff674aafb6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:19.473749 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-818eb431-5962-4cd9-bd97-edff674aafb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:19.486590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:19.487846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:19.488944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:19.488944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:19.488944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:19.488944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:19.488944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:19.488944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:19.488944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:19.488944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:19.495259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 416/799] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:19.496435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eb431-5962-4cd9-bd97-edff674aafb6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.581129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39990200-1225-4ee1-b8a1-c27d8506a02a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:19.618533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f56a3-b684-47d8-bd04-8ec25e73b209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:19.619472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f56a3-b684-47d8-bd04-8ec25e73b209 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:19.620004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f56a3-b684-47d8-bd04-8ec25e73b209 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:19.620483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f56a3-b684-47d8-bd04-8ec25e73b209 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:19.621421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f56a3-b684-47d8-bd04-8ec25e73b209 None admin] RBAC: Adding query filter params (name=network) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:19.621898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f56a3-b684-47d8-bd04-8ec25e73b209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:19.623303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eb431-5962-4cd9-bd97-edff674aafb6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:19.623741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f56a3-b684-47d8-bd04-8ec25e73b209 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:19.630219 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 384/800] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:19.631391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f56a3-b684-47d8-bd04-8ec25e73b209 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.671294 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6da6be-6e0f-4a35-9243-9ac25439271e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:19.671662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6da6be-6e0f-4a35-9243-9ac25439271e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:19.672274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6da6be-6e0f-4a35-9243-9ac25439271e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:19.672862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6da6be-6e0f-4a35-9243-9ac25439271e None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:19.674312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6da6be-6e0f-4a35-9243-9ac25439271e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:19.674958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6da6be-6e0f-4a35-9243-9ac25439271e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:19.676814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6da6be-6e0f-4a35-9243-9ac25439271e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:19.695876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 417/801] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:19.695876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6da6be-6e0f-4a35-9243-9ac25439271e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.727283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:19.743985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:19.754611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f56a3-b684-47d8-bd04-8ec25e73b209 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:19.794776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fac5e1-a5fa-4ccd-b2e2-12414719e568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:19.795962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fac5e1-a5fa-4ccd-b2e2-12414719e568 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:19.796487 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fac5e1-a5fa-4ccd-b2e2-12414719e568 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:19.797008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fac5e1-a5fa-4ccd-b2e2-12414719e568 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:19.798148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fac5e1-a5fa-4ccd-b2e2-12414719e568 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:19.799218 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fac5e1-a5fa-4ccd-b2e2-12414719e568 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:19.801056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fac5e1-a5fa-4ccd-b2e2-12414719e568 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:19.812436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6da6be-6e0f-4a35-9243-9ac25439271e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:19.823066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 385/802] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:19.823066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fac5e1-a5fa-4ccd-b2e2-12414719e568 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.867500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416b4f-4c2b-4d8a-878b-2669cca179f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:19.869048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416b4f-4c2b-4d8a-878b-2669cca179f4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:19.869808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416b4f-4c2b-4d8a-878b-2669cca179f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:19.870378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416b4f-4c2b-4d8a-878b-2669cca179f4 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:19.872014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66416b4f-4c2b-4d8a-878b-2669cca179f4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:19.872835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66416b4f-4c2b-4d8a-878b-2669cca179f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:19.874457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66416b4f-4c2b-4d8a-878b-2669cca179f4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:19.883317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 418/803] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:19.884575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416b4f-4c2b-4d8a-878b-2669cca179f4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:19.976218 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fac5e1-a5fa-4ccd-b2e2-12414719e568 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:20.024354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1668d9-4447-40f9-a067-d3d705ff5f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:20.025615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1668d9-4447-40f9-a067-d3d705ff5f1b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:20.025747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1668d9-4447-40f9-a067-d3d705ff5f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:20.026315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1668d9-4447-40f9-a067-d3d705ff5f1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:20.035511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:20.044036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1668d9-4447-40f9-a067-d3d705ff5f1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:20.046740 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1668d9-4447-40f9-a067-d3d705ff5f1b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:20.048093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416b4f-4c2b-4d8a-878b-2669cca179f4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:20.067964 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-3f1668d9-4447-40f9-a067-d3d705ff5f1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:20.072682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:20.073680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:16:20.074721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:20.077060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 386/804] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:20.078073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1668d9-4447-40f9-a067-d3d705ff5f1b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:20.109731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012dac51-832f-4b97-ae33-dc6846d6780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:20.111120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dac51-832f-4b97-ae33-dc6846d6780d None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:20.112183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dac51-832f-4b97-ae33-dc6846d6780d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:20.112670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dac51-832f-4b97-ae33-dc6846d6780d None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/8731ac7e5a7941f1b2ad04bddc966db6/roles/ee8bde816f274202886522710bb8fdf2` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:20.132922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012dac51-832f-4b97-ae33-dc6846d6780d None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=8731ac7e5a7941f1b2ad04bddc966db6, role_id=ee8bde816f274202886522710bb8fdf2)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:20.134837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012dac51-832f-4b97-ae33-dc6846d6780d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:20.235943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 419/805] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:16:19 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/8731ac7e5a7941f1b2ad04bddc966db6/roles/ee8bde816f274202886522710bb8fdf2 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:16:20.237466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012dac51-832f-4b97-ae33-dc6846d6780d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:20.297948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1668d9-4447-40f9-a067-d3d705ff5f1b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:20.379868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31a9c6-5ceb-47fa-8a84-de2ea2147995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:20.380792 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31a9c6-5ceb-47fa-8a84-de2ea2147995 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:20.381192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31a9c6-5ceb-47fa-8a84-de2ea2147995 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:20.381694 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31a9c6-5ceb-47fa-8a84-de2ea2147995 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:20.382615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31a9c6-5ceb-47fa-8a84-de2ea2147995 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:20.384478 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31a9c6-5ceb-47fa-8a84-de2ea2147995 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:20.420058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012dac51-832f-4b97-ae33-dc6846d6780d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:20.474205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b3f5a-f13c-4a27-bbe1-ca4b80987961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:20.475402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3f5a-f13c-4a27-bbe1-ca4b80987961 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:20.475900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3f5a-f13c-4a27-bbe1-ca4b80987961 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:20.476366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3f5a-f13c-4a27-bbe1-ca4b80987961 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:20.477849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3f5a-f13c-4a27-bbe1-ca4b80987961 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:20.478386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3f5a-f13c-4a27-bbe1-ca4b80987961 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:20.481420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3f5a-f13c-4a27-bbe1-ca4b80987961 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:20.518798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 420/806] 10.222.0.64 () {56 vars in 1135 bytes} [Mon Jun 27 09:16:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:20.523567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbf782-99fd-41a1-8503-737eabe0693a None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:20.523567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbf782-99fd-41a1-8503-737eabe0693a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:20.523567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbf782-99fd-41a1-8503-737eabe0693a None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:20.525494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 421/807] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:20.526995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbf782-99fd-41a1-8503-737eabe0693a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:20.544604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 387/808] 10.222.0.64 () {60 vars in 1144 bytes} [Mon Jun 27 09:16:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:16:20.547388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7caadf-4186-4cc9-9ceb-37bd236b6e19 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:20.548285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7caadf-4186-4cc9-9ceb-37bd236b6e19 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:20.549422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7caadf-4186-4cc9-9ceb-37bd236b6e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:20.555708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:20.574790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ec7caadf-4186-4cc9-9ceb-37bd236b6e19 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:20.577058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ec7caadf-4186-4cc9-9ceb-37bd236b6e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:20.648268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:20.690521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 388/809] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:20.718727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7caadf-4186-4cc9-9ceb-37bd236b6e19 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:20.823195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbf782-99fd-41a1-8503-737eabe0693a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:20.858133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7caadf-4186-4cc9-9ceb-37bd236b6e19 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:20.910486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d471b-cd7f-4955-b13d-9768c389b6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:20.911722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d471b-cd7f-4955-b13d-9768c389b6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:20.912221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d471b-cd7f-4955-b13d-9768c389b6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:20.912724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d471b-cd7f-4955-b13d-9768c389b6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:20.914961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b1f7f-8402-4cf5-a95e-687c5fbdabf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:20.915820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b1f7f-8402-4cf5-a95e-687c5fbdabf2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:20.916135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b1f7f-8402-4cf5-a95e-687c5fbdabf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:20.916413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b1f7f-8402-4cf5-a95e-687c5fbdabf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:20.926349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d471b-cd7f-4955-b13d-9768c389b6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:20.928317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d471b-cd7f-4955-b13d-9768c389b6a0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:20.928728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b1f7f-8402-4cf5-a95e-687c5fbdabf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-f32b1f7f-8402-4cf5-a95e-687c5fbdabf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:20.937154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:20.938158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:20.939066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 389/810] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:20.948284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b1f7f-8402-4cf5-a95e-687c5fbdabf2 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-f48d471b-cd7f-4955-b13d-9768c389b6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:20.981848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:20.983606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:20.985383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:20.985383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:20.985383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:20.985383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:20.991408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 422/811] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 465 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:21.001737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d471b-cd7f-4955-b13d-9768c389b6a0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:21.057315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:21.103402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:21.128647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b1f7f-8402-4cf5-a95e-687c5fbdabf2 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:21.163389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d471b-cd7f-4955-b13d-9768c389b6a0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:21.172249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c1d37-830f-4975-8843-f86890780bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:21.172937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c1d37-830f-4975-8843-f86890780bfc None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.173224 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c1d37-830f-4975-8843-f86890780bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.173501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c1d37-830f-4975-8843-f86890780bfc None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.174550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c1d37-830f-4975-8843-f86890780bfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:21.174860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c1d37-830f-4975-8843-f86890780bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:21.176340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c1d37-830f-4975-8843-f86890780bfc None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:21.186372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 390/812] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:16:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:21.192367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c1d37-830f-4975-8843-f86890780bfc None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:21.220850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94967098-2b8a-4ce4-a149-767b5ed79aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:21.222100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94967098-2b8a-4ce4-a149-767b5ed79aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.222488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94967098-2b8a-4ce4-a149-767b5ed79aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.222899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94967098-2b8a-4ce4-a149-767b5ed79aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.223778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94967098-2b8a-4ce4-a149-767b5ed79aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:21.224195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94967098-2b8a-4ce4-a149-767b5ed79aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:21.226370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94967098-2b8a-4ce4-a149-767b5ed79aa5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:21.243569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 423/813] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:21.249438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94967098-2b8a-4ce4-a149-767b5ed79aa5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:21.365319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:21.371182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c1d37-830f-4975-8843-f86890780bfc None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:21.390405 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94967098-2b8a-4ce4-a149-767b5ed79aa5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:21.419583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a45e9b-bd1b-44b9-b5fc-00e3865c795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:21.419909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:21.420660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a45e9b-bd1b-44b9-b5fc-00e3865c795c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.420834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a45e9b-bd1b-44b9-b5fc-00e3865c795c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.421225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a45e9b-bd1b-44b9-b5fc-00e3865c795c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.439290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a45e9b-bd1b-44b9-b5fc-00e3865c795c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:21.441635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a45e9b-bd1b-44b9-b5fc-00e3865c795c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:21.443344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e794043-0e76-4d47-9a98-cfa8619f8f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:21.444878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794043-0e76-4d47-9a98-cfa8619f8f3d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.445346 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794043-0e76-4d47-9a98-cfa8619f8f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.445825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794043-0e76-4d47-9a98-cfa8619f8f3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.454531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e794043-0e76-4d47-9a98-cfa8619f8f3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:21.456314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e794043-0e76-4d47-9a98-cfa8619f8f3d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-70a45e9b-bd1b-44b9-b5fc-00e3865c795c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:21.461308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:21.462244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:21.463133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:21.463133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:21.463133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:21.463133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:21.467194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 391/814] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:21.468406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254b0a2-3c71-4205-9222-b3afb99580af None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.468698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254b0a2-3c71-4205-9222-b3afb99580af None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.468977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254b0a2-3c71-4205-9222-b3afb99580af None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.471644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 392/815] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:21.471644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254b0a2-3c71-4205-9222-b3afb99580af None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-9e794043-0e76-4d47-9a98-cfa8619f8f3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:21.473767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:21.475145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:21.476287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:21.476287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:21.476287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:21.476287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:21.476287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:21.476287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:21.476287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:21.476287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:21.479874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 424/816] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:21.481179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6985f8-8a40-45da-a68f-b9b945af2e12 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.481461 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6985f8-8a40-45da-a68f-b9b945af2e12 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.481739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6985f8-8a40-45da-a68f-b9b945af2e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.499423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8f6985f8-8a40-45da-a68f-b9b945af2e12 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:21.500647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8f6985f8-8a40-45da-a68f-b9b945af2e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:21.590311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 425/817] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:21.591459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6985f8-8a40-45da-a68f-b9b945af2e12 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:21.626724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254b0a2-3c71-4205-9222-b3afb99580af None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:21.678501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef8d0b-49de-40af-bb22-f4250e8bd25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:21.679086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef8d0b-49de-40af-bb22-f4250e8bd25a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.679521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef8d0b-49de-40af-bb22-f4250e8bd25a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.679679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef8d0b-49de-40af-bb22-f4250e8bd25a None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.680405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef8d0b-49de-40af-bb22-f4250e8bd25a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:21.680681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef8d0b-49de-40af-bb22-f4250e8bd25a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:21.682065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef8d0b-49de-40af-bb22-f4250e8bd25a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:21.694024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 393/818] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:21.694848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef8d0b-49de-40af-bb22-f4250e8bd25a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:21.710782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:21.726265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6985f8-8a40-45da-a68f-b9b945af2e12 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:21.744939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:21.772224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484ad9a-f5eb-4db0-a8a4-3fe3adf2a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:21.772923 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484ad9a-f5eb-4db0-a8a4-3fe3adf2a63b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.773260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484ad9a-f5eb-4db0-a8a4-3fe3adf2a63b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.773538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484ad9a-f5eb-4db0-a8a4-3fe3adf2a63b None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.774404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484ad9a-f5eb-4db0-a8a4-3fe3adf2a63b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:21.774728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484ad9a-f5eb-4db0-a8a4-3fe3adf2a63b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:21.776488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484ad9a-f5eb-4db0-a8a4-3fe3adf2a63b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:21.794581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 426/819] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:21.795950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484ad9a-f5eb-4db0-a8a4-3fe3adf2a63b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:21.819471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef8d0b-49de-40af-bb22-f4250e8bd25a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:21.866586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d1adf-5444-41fd-b8b9-0ff2b5a5f440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:21.867399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d1adf-5444-41fd-b8b9-0ff2b5a5f440 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.867708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d1adf-5444-41fd-b8b9-0ff2b5a5f440 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.868034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d1adf-5444-41fd-b8b9-0ff2b5a5f440 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.868631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d1adf-5444-41fd-b8b9-0ff2b5a5f440 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:21.870014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d1adf-5444-41fd-b8b9-0ff2b5a5f440 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-7a7d1adf-5444-41fd-b8b9-0ff2b5a5f440 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:21.884085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:21.885212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 09:16:21.886194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:21.893342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 394/820] 10.222.0.64 () {56 vars in 1129 bytes} [Mon Jun 27 09:16:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:21.894351 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d1adf-5444-41fd-b8b9-0ff2b5a5f440 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:21.921573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484ad9a-f5eb-4db0-a8a4-3fe3adf2a63b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:21.961583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2bf0b-03f1-4652-983c-c86bf4cdf1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:21.962304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2bf0b-03f1-4652-983c-c86bf4cdf1bb None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:21.962583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2bf0b-03f1-4652-983c-c86bf4cdf1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:21.962856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2bf0b-03f1-4652-983c-c86bf4cdf1bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:21.972552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2bf0b-03f1-4652-983c-c86bf4cdf1bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:21.974055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2bf0b-03f1-4652-983c-c86bf4cdf1bb None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3dd2bf0b-03f1-4652-983c-c86bf4cdf1bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:21.992706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:21.993740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 427/821] 10.222.0.64 () {56 vars in 1093 bytes} [Mon Jun 27 09:16:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:21.995359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2bf0b-03f1-4652-983c-c86bf4cdf1bb None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:22.006404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:22.035578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d1adf-5444-41fd-b8b9-0ff2b5a5f440 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:22.086841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea39210-db4a-45bf-b884-16dd0be4a79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:22.087796 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea39210-db4a-45bf-b884-16dd0be4a79e None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:22.088091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea39210-db4a-45bf-b884-16dd0be4a79e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:22.088368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea39210-db4a-45bf-b884-16dd0be4a79e None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/70b499a32516486d9a2bed34f7e66b76/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:22.090532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea39210-db4a-45bf-b884-16dd0be4a79e None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=70b499a32516486d9a2bed34f7e66b76, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:22.092073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea39210-db4a-45bf-b884-16dd0be4a79e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:22.158044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:22.176142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 395/822] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:16:21 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/70b499a32516486d9a2bed34f7e66b76/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:16:22.178008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea39210-db4a-45bf-b884-16dd0be4a79e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:22.276436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2bf0b-03f1-4652-983c-c86bf4cdf1bb None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:22.338537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18c0b8-dd7e-4b83-bbd5-a109e0e559fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:22.339565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18c0b8-dd7e-4b83-bbd5-a109e0e559fc None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:22.340060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18c0b8-dd7e-4b83-bbd5-a109e0e559fc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:22.340515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18c0b8-dd7e-4b83-bbd5-a109e0e559fc None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:22.341421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18c0b8-dd7e-4b83-bbd5-a109e0e559fc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:22.341906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18c0b8-dd7e-4b83-bbd5-a109e0e559fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:22.343500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18c0b8-dd7e-4b83-bbd5-a109e0e559fc None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:22.356311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 428/823] 10.222.0.64 () {56 vars in 1124 bytes} [Mon Jun 27 09:16:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:22.357754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18c0b8-dd7e-4b83-bbd5-a109e0e559fc None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:22.462857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea39210-db4a-45bf-b884-16dd0be4a79e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:22.511307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:22.512119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6e919-e5da-4cba-995d-83729b1d9c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:22.512119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6e919-e5da-4cba-995d-83729b1d9c8d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:22.512450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6e919-e5da-4cba-995d-83729b1d9c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:22.512584 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6e919-e5da-4cba-995d-83729b1d9c8d None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:22.514331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6e919-e5da-4cba-995d-83729b1d9c8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:22.514832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6e919-e5da-4cba-995d-83729b1d9c8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:22.516862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6e919-e5da-4cba-995d-83729b1d9c8d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:22.520556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:22.528598 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18c0b8-dd7e-4b83-bbd5-a109e0e559fc None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:22.543471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 396/824] 10.222.0.64 () {56 vars in 1133 bytes} [Mon Jun 27 09:16:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:22.550426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6e919-e5da-4cba-995d-83729b1d9c8d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:22.592441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607710a-dd28-45e0-a275-745f0baabc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:22.593602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607710a-dd28-45e0-a275-745f0baabc8f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:22.594083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607710a-dd28-45e0-a275-745f0baabc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:22.594620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607710a-dd28-45e0-a275-745f0baabc8f None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:22.596031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607710a-dd28-45e0-a275-745f0baabc8f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:22.596679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607710a-dd28-45e0-a275-745f0baabc8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:22.598999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607710a-dd28-45e0-a275-745f0baabc8f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:22.609924 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 429/825] 10.222.0.64 () {56 vars in 1124 bytes} [Mon Jun 27 09:16:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:22.612692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130139-b38e-486c-b6e9-b7876646fa2d None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:22.613246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130139-b38e-486c-b6e9-b7876646fa2d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:22.614160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130139-b38e-486c-b6e9-b7876646fa2d None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:22.620173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 430/826] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:22.621484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130139-b38e-486c-b6e9-b7876646fa2d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:22.690783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6e919-e5da-4cba-995d-83729b1d9c8d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:22.732179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e9552-bdf8-41d8-bdac-8416f90ab2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:22.732860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e9552-bdf8-41d8-bdac-8416f90ab2ad None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:22.733133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e9552-bdf8-41d8-bdac-8416f90ab2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:22.733604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e9552-bdf8-41d8-bdac-8416f90ab2ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:22.794266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130139-b38e-486c-b6e9-b7876646fa2d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:22.832636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee3f83-d6f8-42c4-93cc-c3b304ede624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:22.833291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee3f83-d6f8-42c4-93cc-c3b304ede624 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:22.833573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee3f83-d6f8-42c4-93cc-c3b304ede624 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:22.833843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee3f83-d6f8-42c4-93cc-c3b304ede624 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:22.834595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee3f83-d6f8-42c4-93cc-c3b304ede624 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:22.835097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:22.836043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee3f83-d6f8-42c4-93cc-c3b304ede624 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:22.847807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e9552-bdf8-41d8-bdac-8416f90ab2ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:22.854567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e9552-bdf8-41d8-bdac-8416f90ab2ad None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:22.931564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 431/827] 10.222.0.64 () {60 vars in 1144 bytes} [Mon Jun 27 09:16:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:16:22.931564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f7807-042c-492e-b5e8-1d8dacf32870 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:22.931564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f7807-042c-492e-b5e8-1d8dacf32870 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:22.931564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f7807-042c-492e-b5e8-1d8dacf32870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:22.964136 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-415f7807-042c-492e-b5e8-1d8dacf32870 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:22.966105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-415f7807-042c-492e-b5e8-1d8dacf32870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:22.997950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:23.056151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 397/828] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:22 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:16:23.065418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e9552-bdf8-41d8-bdac-8416f90ab2ad None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:23.102776 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 432/829] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:23.129271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f7807-042c-492e-b5e8-1d8dacf32870 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:23.192553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:23.203223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e9552-bdf8-41d8-bdac-8416f90ab2ad None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:23.252241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346de6d1-a40d-4087-85e5-a5786ad19673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:23.253285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346de6d1-a40d-4087-85e5-a5786ad19673 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:23.253951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346de6d1-a40d-4087-85e5-a5786ad19673 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:23.254234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346de6d1-a40d-4087-85e5-a5786ad19673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:23.256482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f7807-042c-492e-b5e8-1d8dacf32870 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:23.268941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346de6d1-a40d-4087-85e5-a5786ad19673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:23.271063 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346de6d1-a40d-4087-85e5-a5786ad19673 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-346de6d1-a40d-4087-85e5-a5786ad19673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:23.299559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:23.300541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:23.301451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:23.301451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:23.301451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:23.301451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:23.303621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 398/830] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:23.305927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346de6d1-a40d-4087-85e5-a5786ad19673 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:23.308397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca459c-f801-4897-990a-9e2776536e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:23.309521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca459c-f801-4897-990a-9e2776536e1e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:23.309920 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca459c-f801-4897-990a-9e2776536e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:23.311216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca459c-f801-4897-990a-9e2776536e1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:23.325311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca459c-f801-4897-990a-9e2776536e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-adca459c-f801-4897-990a-9e2776536e1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:23.337801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:23.339648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:23.340895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 433/831] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:16:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:23.348832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca459c-f801-4897-990a-9e2776536e1e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:23.367939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:23.481711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346de6d1-a40d-4087-85e5-a5786ad19673 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:23.497563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca459c-f801-4897-990a-9e2776536e1e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:23.529198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26827a8a-32ba-4d8b-bd3b-947360c67d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:23.530012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26827a8a-32ba-4d8b-bd3b-947360c67d23 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:23.530369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26827a8a-32ba-4d8b-bd3b-947360c67d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:23.530722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26827a8a-32ba-4d8b-bd3b-947360c67d23 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:23.531916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26827a8a-32ba-4d8b-bd3b-947360c67d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:23.532339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26827a8a-32ba-4d8b-bd3b-947360c67d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:23.535161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26827a8a-32ba-4d8b-bd3b-947360c67d23 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:23.550007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d286bc-e5ee-4487-b2a1-7f2854a7536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:23.551407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d286bc-e5ee-4487-b2a1-7f2854a7536e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:23.552167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d286bc-e5ee-4487-b2a1-7f2854a7536e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:23.552734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d286bc-e5ee-4487-b2a1-7f2854a7536e None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:23.554503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d286bc-e5ee-4487-b2a1-7f2854a7536e None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:23.555161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d286bc-e5ee-4487-b2a1-7f2854a7536e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:23.556609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 399/832] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:23.557925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26827a8a-32ba-4d8b-bd3b-947360c67d23 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:23.558277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d286bc-e5ee-4487-b2a1-7f2854a7536e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:23.568662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:23.576212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 434/833] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:16:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:23.582645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d286bc-e5ee-4487-b2a1-7f2854a7536e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:23.698791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:23.700201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26827a8a-32ba-4d8b-bd3b-947360c67d23 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:23.741454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493e111-13f4-49cd-94b1-2fc75e2be94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:23.742261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493e111-13f4-49cd-94b1-2fc75e2be94f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:23.742606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493e111-13f4-49cd-94b1-2fc75e2be94f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:23.742879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493e111-13f4-49cd-94b1-2fc75e2be94f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:23.751255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493e111-13f4-49cd-94b1-2fc75e2be94f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:23.752907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493e111-13f4-49cd-94b1-2fc75e2be94f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:23.759282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d286bc-e5ee-4487-b2a1-7f2854a7536e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-6493e111-13f4-49cd-94b1-2fc75e2be94f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:23.769189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:23.770308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:23.771273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:23.771273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:23.771273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:23.771273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:23.771273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:23.771273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:23.771273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:23.771273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:23.775654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 400/834] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:23.777339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493e111-13f4-49cd-94b1-2fc75e2be94f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:23.812591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f243190-1142-4738-90b8-cb4fdbdfe2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:23.813950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f243190-1142-4738-90b8-cb4fdbdfe2ab None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:23.814446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f243190-1142-4738-90b8-cb4fdbdfe2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:23.815187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f243190-1142-4738-90b8-cb4fdbdfe2ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:23.828796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f243190-1142-4738-90b8-cb4fdbdfe2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:23.830965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f243190-1142-4738-90b8-cb4fdbdfe2ab None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3f243190-1142-4738-90b8-cb4fdbdfe2ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:23.850738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:23.851980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:23.853090 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:23.853090 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:23.853090 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:23.853090 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:23.854903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 435/835] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:23.864297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f243190-1142-4738-90b8-cb4fdbdfe2ab None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:23.868296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:23.918302 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493e111-13f4-49cd-94b1-2fc75e2be94f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:23.979093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86b270-ff8a-4a3a-8671-4ee45bfaee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:23.979820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86b270-ff8a-4a3a-8671-4ee45bfaee4a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:23.980103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86b270-ff8a-4a3a-8671-4ee45bfaee4a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:23.980382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86b270-ff8a-4a3a-8671-4ee45bfaee4a None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:23.981201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86b270-ff8a-4a3a-8671-4ee45bfaee4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:23.981512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86b270-ff8a-4a3a-8671-4ee45bfaee4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:23.983332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86b270-ff8a-4a3a-8671-4ee45bfaee4a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:23.998368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 401/836] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:24.004418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86b270-ff8a-4a3a-8671-4ee45bfaee4a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:24.088177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f243190-1142-4738-90b8-cb4fdbdfe2ab None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:24.117537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:24.153931 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba895b6-ce3a-4197-9d6f-0c698c0c29e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:24.155510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba895b6-ce3a-4197-9d6f-0c698c0c29e4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:24.156479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba895b6-ce3a-4197-9d6f-0c698c0c29e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:24.157349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba895b6-ce3a-4197-9d6f-0c698c0c29e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:24.159113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba895b6-ce3a-4197-9d6f-0c698c0c29e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:24.160046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba895b6-ce3a-4197-9d6f-0c698c0c29e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:24.163279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba895b6-ce3a-4197-9d6f-0c698c0c29e4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:24.193893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 436/837] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:24.199282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba895b6-ce3a-4197-9d6f-0c698c0c29e4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:24.226293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86b270-ff8a-4a3a-8671-4ee45bfaee4a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:24.265442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:24.277362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85cd0c-b1f1-40fe-9cee-39badfa5ee70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:24.278182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85cd0c-b1f1-40fe-9cee-39badfa5ee70 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:24.278625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85cd0c-b1f1-40fe-9cee-39badfa5ee70 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:24.280228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85cd0c-b1f1-40fe-9cee-39badfa5ee70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:24.280228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85cd0c-b1f1-40fe-9cee-39badfa5ee70 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19e0fbe0b5814ad1ae95c65a64b632d6) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:24.280228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85cd0c-b1f1-40fe-9cee-39badfa5ee70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:24.282027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85cd0c-b1f1-40fe-9cee-39badfa5ee70 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:24.295569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 402/838] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:24 2022] GET /identity/v3/role_assignments?user.id=19e0fbe0b5814ad1ae95c65a64b632d6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=03b724e03f704f2ab338b64025a3db6d => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:24.381918 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba895b6-ce3a-4197-9d6f-0c698c0c29e4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:24.450956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62ba99-b142-413a-961d-e3b49cd3a04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:24.452865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62ba99-b142-413a-961d-e3b49cd3a04f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:24.453647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62ba99-b142-413a-961d-e3b49cd3a04f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:24.454420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62ba99-b142-413a-961d-e3b49cd3a04f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:24.470504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62ba99-b142-413a-961d-e3b49cd3a04f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:24.473197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62ba99-b142-413a-961d-e3b49cd3a04f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-ea62ba99-b142-413a-961d-e3b49cd3a04f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:24.493348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:24.494366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:24.495322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:24.497826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 437/839] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:16:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:24.502915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10a44f-a089-4b0b-972f-31ecb7ae2ef5 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:24.503227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10a44f-a089-4b0b-972f-31ecb7ae2ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:24.503512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10a44f-a089-4b0b-972f-31ecb7ae2ef5 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:24.504518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62ba99-b142-413a-961d-e3b49cd3a04f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:24.505156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 403/840] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:24.510675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb5f7b-0d62-4c97-837d-56ed68d4238f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:24.510949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb5f7b-0d62-4c97-837d-56ed68d4238f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:24.511237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb5f7b-0d62-4c97-837d-56ed68d4238f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:24.535347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-31fb5f7b-0d62-4c97-837d-56ed68d4238f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:24.536631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-31fb5f7b-0d62-4c97-837d-56ed68d4238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:24.565838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:24.644027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 404/841] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:24.668647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb5f7b-0d62-4c97-837d-56ed68d4238f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:24.677411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62ba99-b142-413a-961d-e3b49cd3a04f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:24.728702 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13348bf3-d0e2-4b32-b3aa-26c010c39f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:24.729824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13348bf3-d0e2-4b32-b3aa-26c010c39f22 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:24.731595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13348bf3-d0e2-4b32-b3aa-26c010c39f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:24.732755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13348bf3-d0e2-4b32-b3aa-26c010c39f22 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:24.733764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13348bf3-d0e2-4b32-b3aa-26c010c39f22 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:24.734881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13348bf3-d0e2-4b32-b3aa-26c010c39f22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:24.738376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13348bf3-d0e2-4b32-b3aa-26c010c39f22 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:24.768304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 438/842] 10.222.0.64 () {56 vars in 1149 bytes} [Mon Jun 27 09:16:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:24.772237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13348bf3-d0e2-4b32-b3aa-26c010c39f22 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:24.814280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb5f7b-0d62-4c97-837d-56ed68d4238f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:24.853545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:24.881754 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43c962-a03a-4c6b-8627-2d071eb9e8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:24.883281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43c962-a03a-4c6b-8627-2d071eb9e8bc None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:24.883878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43c962-a03a-4c6b-8627-2d071eb9e8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:24.884494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43c962-a03a-4c6b-8627-2d071eb9e8bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:24.888059 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:24.902939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13348bf3-d0e2-4b32-b3aa-26c010c39f22 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:24.905532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43c962-a03a-4c6b-8627-2d071eb9e8bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-4a43c962-a03a-4c6b-8627-2d071eb9e8bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:24.919484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:24.920655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:24.921876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:24.923242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 405/843] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:24.933835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43c962-a03a-4c6b-8627-2d071eb9e8bc None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:24.950875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ff486-5adf-4974-aa13-77d5e5a4e1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:24.952355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ff486-5adf-4974-aa13-77d5e5a4e1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:24.953002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ff486-5adf-4974-aa13-77d5e5a4e1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:24.953642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ff486-5adf-4974-aa13-77d5e5a4e1b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:25.042942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ff486-5adf-4974-aa13-77d5e5a4e1b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:25.044559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ff486-5adf-4974-aa13-77d5e5a4e1b1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:25.087601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43c962-a03a-4c6b-8627-2d071eb9e8bc None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:25.136926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93064e-4793-4bf4-8aad-8543a34b57ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:25.137648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93064e-4793-4bf4-8aad-8543a34b57ba None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:25.138140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93064e-4793-4bf4-8aad-8543a34b57ba None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:25.138343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93064e-4793-4bf4-8aad-8543a34b57ba None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:25.139771 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93064e-4793-4bf4-8aad-8543a34b57ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:25.140083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93064e-4793-4bf4-8aad-8543a34b57ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:25.142032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93064e-4793-4bf4-8aad-8543a34b57ba None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:25.155178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 406/844] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:16:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:25.155898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:25.159688 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 439/845] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:24 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:16:25.161240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93064e-4793-4bf4-8aad-8543a34b57ba None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:25.168647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ff486-5adf-4974-aa13-77d5e5a4e1b1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:25.270518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:25.326002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ff486-5adf-4974-aa13-77d5e5a4e1b1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:25.362214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93064e-4793-4bf4-8aad-8543a34b57ba None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:25.372674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a253ac1-5009-44cc-ba3a-e1a5ecd5ade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:25.373583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a253ac1-5009-44cc-ba3a-e1a5ecd5ade9 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:25.373936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a253ac1-5009-44cc-ba3a-e1a5ecd5ade9 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:25.374276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a253ac1-5009-44cc-ba3a-e1a5ecd5ade9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:25.386422 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a253ac1-5009-44cc-ba3a-e1a5ecd5ade9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:25.387926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a253ac1-5009-44cc-ba3a-e1a5ecd5ade9 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:25.413030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371353e-7d44-435e-a6fa-b2a8e01fef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:25.413749 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371353e-7d44-435e-a6fa-b2a8e01fef39 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:25.414030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371353e-7d44-435e-a6fa-b2a8e01fef39 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:25.414532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371353e-7d44-435e-a6fa-b2a8e01fef39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3a253ac1-5009-44cc-ba3a-e1a5ecd5ade9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:25.415443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:25.416569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:25.417651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:25.417651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:25.417651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:25.417651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:25.417651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 440/846] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:25.418844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e725f6-c577-44d4-b337-fcad4f4857d2 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:25.419176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e725f6-c577-44d4-b337-fcad4f4857d2 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:25.419788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e725f6-c577-44d4-b337-fcad4f4857d2 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:25.427776 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 441/847] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:25.427776 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e725f6-c577-44d4-b337-fcad4f4857d2 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:25.441050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371353e-7d44-435e-a6fa-b2a8e01fef39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:25.442534 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371353e-7d44-435e-a6fa-b2a8e01fef39 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-b371353e-7d44-435e-a6fa-b2a8e01fef39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:25.463111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:25.464255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:25.466301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:25.466301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:25.466301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:25.466301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:25.466301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 407/848] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:25.466301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b311a1-db8b-4f85-85bb-3b30f7bdbe8f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:25.466301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b311a1-db8b-4f85-85bb-3b30f7bdbe8f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:25.466301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b311a1-db8b-4f85-85bb-3b30f7bdbe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:25.484922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f6b311a1-db8b-4f85-85bb-3b30f7bdbe8f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:25.486161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f6b311a1-db8b-4f85-85bb-3b30f7bdbe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:25.512900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:25.579553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 408/849] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:25.579553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b311a1-db8b-4f85-85bb-3b30f7bdbe8f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:25.584785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e725f6-c577-44d4-b337-fcad4f4857d2 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:25.619741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:25.628620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cfd76-616c-48ce-85d4-fadd98bdb2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:25.629497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cfd76-616c-48ce-85d4-fadd98bdb2cb None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:25.629768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cfd76-616c-48ce-85d4-fadd98bdb2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:25.629967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cfd76-616c-48ce-85d4-fadd98bdb2cb None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:25.630712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cfd76-616c-48ce-85d4-fadd98bdb2cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:25.631012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cfd76-616c-48ce-85d4-fadd98bdb2cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:25.632457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cfd76-616c-48ce-85d4-fadd98bdb2cb None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:25.652131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 442/850] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:25.652959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cfd76-616c-48ce-85d4-fadd98bdb2cb None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:25.707103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b311a1-db8b-4f85-85bb-3b30f7bdbe8f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:25.762485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f83752-f20e-4d83-98b1-1891f36a8aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:25.763403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f83752-f20e-4d83-98b1-1891f36a8aea None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:25.763748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f83752-f20e-4d83-98b1-1891f36a8aea None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:25.764052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f83752-f20e-4d83-98b1-1891f36a8aea None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:25.764818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f83752-f20e-4d83-98b1-1891f36a8aea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:25.765141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f83752-f20e-4d83-98b1-1891f36a8aea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:25.766629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f83752-f20e-4d83-98b1-1891f36a8aea None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:25.782891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cfd76-616c-48ce-85d4-fadd98bdb2cb None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:25.786748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 409/851] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:25.787778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f83752-f20e-4d83-98b1-1891f36a8aea None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:25.791408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:25.829982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e63ed9-f09a-4d25-853d-09b1d684a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:25.831571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e63ed9-f09a-4d25-853d-09b1d684a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:25.831571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e63ed9-f09a-4d25-853d-09b1d684a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:25.831769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e63ed9-f09a-4d25-853d-09b1d684a2a0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:25.832419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e63ed9-f09a-4d25-853d-09b1d684a2a0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:25.834494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e63ed9-f09a-4d25-853d-09b1d684a2a0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-c0e63ed9-f09a-4d25-853d-09b1d684a2a0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:25.849473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:25.850522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:25.851347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 443/852] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:16:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:25.852168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e63ed9-f09a-4d25-853d-09b1d684a2a0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:25.917666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:25.947448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f83752-f20e-4d83-98b1-1891f36a8aea None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:25.988136 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e63ed9-f09a-4d25-853d-09b1d684a2a0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:26.005192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d5596-acb0-40f1-8fe6-b3b45e55f7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:26.006468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d5596-acb0-40f1-8fe6-b3b45e55f7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.006949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d5596-acb0-40f1-8fe6-b3b45e55f7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.007418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d5596-acb0-40f1-8fe6-b3b45e55f7b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.020408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d5596-acb0-40f1-8fe6-b3b45e55f7b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:26.022927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d5596-acb0-40f1-8fe6-b3b45e55f7b0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:26.046387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb5e3d-2b1f-40d0-85a5-2d6f4dfe95c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:26.048201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb5e3d-2b1f-40d0-85a5-2d6f4dfe95c4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-ae8d5596-acb0-40f1-8fe6-b3b45e55f7b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:26.049130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:26.050884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 410/853] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:26.052804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d5596-acb0-40f1-8fe6-b3b45e55f7b0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:26.055695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb5e3d-2b1f-40d0-85a5-2d6f4dfe95c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.056480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb5e3d-2b1f-40d0-85a5-2d6f4dfe95c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.068500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb5e3d-2b1f-40d0-85a5-2d6f4dfe95c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:26.071395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb5e3d-2b1f-40d0-85a5-2d6f4dfe95c4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-e8bb5e3d-2b1f-40d0-85a5-2d6f4dfe95c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:26.095466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:26.096838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 444/854] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:26.098575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb5e3d-2b1f-40d0-85a5-2d6f4dfe95c4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:26.149112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:26.225161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d5596-acb0-40f1-8fe6-b3b45e55f7b0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:26.251801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb5e3d-2b1f-40d0-85a5-2d6f4dfe95c4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:26.264633 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae91dc5-f5da-4323-9c52-7d3f999261c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:26.265382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae91dc5-f5da-4323-9c52-7d3f999261c4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.265660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae91dc5-f5da-4323-9c52-7d3f999261c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.265931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae91dc5-f5da-4323-9c52-7d3f999261c4 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.266693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae91dc5-f5da-4323-9c52-7d3f999261c4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:26.267057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae91dc5-f5da-4323-9c52-7d3f999261c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:26.268531 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae91dc5-f5da-4323-9c52-7d3f999261c4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:26.274449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 411/855] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:16:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:26.275177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae91dc5-f5da-4323-9c52-7d3f999261c4 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:26.289684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:26.309692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b884d-aa10-4a2b-a1fe-fd2b4bbe77c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:26.310436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b884d-aa10-4a2b-a1fe-fd2b4bbe77c4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.310728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b884d-aa10-4a2b-a1fe-fd2b4bbe77c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.311036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b884d-aa10-4a2b-a1fe-fd2b4bbe77c4 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.311916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b884d-aa10-4a2b-a1fe-fd2b4bbe77c4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:26.312234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b884d-aa10-4a2b-a1fe-fd2b4bbe77c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:26.313680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b884d-aa10-4a2b-a1fe-fd2b4bbe77c4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:26.341205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 445/856] 10.222.0.64 () {56 vars in 1135 bytes} [Mon Jun 27 09:16:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:26.341205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b884d-aa10-4a2b-a1fe-fd2b4bbe77c4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:26.406101 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae91dc5-f5da-4323-9c52-7d3f999261c4 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:26.444979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b757f23-4112-4156-805d-1ddba43e429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:26.445647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b757f23-4112-4156-805d-1ddba43e429d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.445935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b757f23-4112-4156-805d-1ddba43e429d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.446212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b757f23-4112-4156-805d-1ddba43e429d None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.447004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b757f23-4112-4156-805d-1ddba43e429d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:26.447320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b757f23-4112-4156-805d-1ddba43e429d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:26.448811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b757f23-4112-4156-805d-1ddba43e429d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:26.468441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 412/857] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:26.468441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b757f23-4112-4156-805d-1ddba43e429d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:26.493198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:26.512127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b884d-aa10-4a2b-a1fe-fd2b4bbe77c4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:26.550813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:26.559975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799d2ec-df68-4d91-9a9d-5d052a9d2c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:26.561362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799d2ec-df68-4d91-9a9d-5d052a9d2c15 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.561871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799d2ec-df68-4d91-9a9d-5d052a9d2c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.568947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799d2ec-df68-4d91-9a9d-5d052a9d2c15 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.568947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799d2ec-df68-4d91-9a9d-5d052a9d2c15 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:26.568947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799d2ec-df68-4d91-9a9d-5d052a9d2c15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:26.568947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799d2ec-df68-4d91-9a9d-5d052a9d2c15 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:26.587562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 446/858] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:16:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:26.587562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799d2ec-df68-4d91-9a9d-5d052a9d2c15 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:26.595131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b757f23-4112-4156-805d-1ddba43e429d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:26.643695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae62c6-87b8-4ed2-b0f9-d089f72806b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:26.644371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae62c6-87b8-4ed2-b0f9-d089f72806b4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.644649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae62c6-87b8-4ed2-b0f9-d089f72806b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.644926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae62c6-87b8-4ed2-b0f9-d089f72806b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.735972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799d2ec-df68-4d91-9a9d-5d052a9d2c15 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:26.743739 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae62c6-87b8-4ed2-b0f9-d089f72806b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:26.745546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae62c6-87b8-4ed2-b0f9-d089f72806b4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:26.803478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167c42c-4ea8-447c-9315-012de4beb1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:26.804594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167c42c-4ea8-447c-9315-012de4beb1ab None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.805114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167c42c-4ea8-447c-9315-012de4beb1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.805627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167c42c-4ea8-447c-9315-012de4beb1ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.807325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167c42c-4ea8-447c-9315-012de4beb1ab None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=8731ac7e5a7941f1b2ad04bddc966db6) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:26.807914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167c42c-4ea8-447c-9315-012de4beb1ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:26.809492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167c42c-4ea8-447c-9315-012de4beb1ab None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:26.816141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 447/859] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:26 2022] GET /identity/v3/role_assignments?user.id=8731ac7e5a7941f1b2ad04bddc966db6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:26.817516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65468b9-883a-47ba-8a2f-e2a85195cc20 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.817813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65468b9-883a-47ba-8a2f-e2a85195cc20 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.818091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65468b9-883a-47ba-8a2f-e2a85195cc20 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.823672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 448/860] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:26.825121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b6f5-1ed6-4149-9912-561be5d90cee None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:26.825271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:26.825419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b6f5-1ed6-4149-9912-561be5d90cee None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:26.825701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b6f5-1ed6-4149-9912-561be5d90cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:26.848668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-0d02b6f5-1ed6-4149-9912-561be5d90cee None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:26.850136 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-0d02b6f5-1ed6-4149-9912-561be5d90cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:26.871682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 413/861] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:26 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:16:26.871949 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:26.878693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae62c6-87b8-4ed2-b0f9-d089f72806b4 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:26.937609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 449/862] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:26.964616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02b6f5-1ed6-4149-9912-561be5d90cee None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:26.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae62c6-87b8-4ed2-b0f9-d089f72806b4 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:27.039688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e7eb-a119-49f2-a228-f9e5c4273bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:27.040504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e7eb-a119-49f2-a228-f9e5c4273bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:27.040785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e7eb-a119-49f2-a228-f9e5c4273bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:27.041062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e7eb-a119-49f2-a228-f9e5c4273bd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:27.052592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e7eb-a119-49f2-a228-f9e5c4273bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:27.054251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e7eb-a119-49f2-a228-f9e5c4273bd0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0317e7eb-a119-49f2-a228-f9e5c4273bd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:27.075076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:27.076237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:27.077336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:27.077336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:27.077336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:27.077336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:27.077336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 414/863] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:27.084060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e7eb-a119-49f2-a228-f9e5c4273bd0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:27.092154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02b6f5-1ed6-4149-9912-561be5d90cee None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:27.108826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:27.139922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303a3ab-4fcd-46c6-8b76-7e0cbf3f98b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:27.140758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303a3ab-4fcd-46c6-8b76-7e0cbf3f98b5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:27.141041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303a3ab-4fcd-46c6-8b76-7e0cbf3f98b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:27.141319 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303a3ab-4fcd-46c6-8b76-7e0cbf3f98b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:27.141989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303a3ab-4fcd-46c6-8b76-7e0cbf3f98b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:27.143993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303a3ab-4fcd-46c6-8b76-7e0cbf3f98b5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-7303a3ab-4fcd-46c6-8b76-7e0cbf3f98b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:27.159012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:27.160194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:27.161343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 450/864] 10.222.0.64 () {56 vars in 1132 bytes} [Mon Jun 27 09:16:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:27.168586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303a3ab-4fcd-46c6-8b76-7e0cbf3f98b5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:27.183857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:27.209726 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e7eb-a119-49f2-a228-f9e5c4273bd0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:27.250678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3069986-d118-40af-ad1c-d6ef7659f9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:27.252229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3069986-d118-40af-ad1c-d6ef7659f9cd None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:27.252700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3069986-d118-40af-ad1c-d6ef7659f9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:27.253122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3069986-d118-40af-ad1c-d6ef7659f9cd None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:27.254096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3069986-d118-40af-ad1c-d6ef7659f9cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:27.254667 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3069986-d118-40af-ad1c-d6ef7659f9cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:27.256365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3069986-d118-40af-ad1c-d6ef7659f9cd None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:27.275623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 415/865] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:27.277505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3069986-d118-40af-ad1c-d6ef7659f9cd None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:27.294438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303a3ab-4fcd-46c6-8b76-7e0cbf3f98b5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:27.338541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272a100-cda7-434c-8e52-e078908eb700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:27.339562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a100-cda7-434c-8e52-e078908eb700 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:27.339829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a100-cda7-434c-8e52-e078908eb700 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:27.340223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272a100-cda7-434c-8e52-e078908eb700 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:27.341053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272a100-cda7-434c-8e52-e078908eb700 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:27.341463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272a100-cda7-434c-8e52-e078908eb700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:27.342944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272a100-cda7-434c-8e52-e078908eb700 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:27.349306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 451/866] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:27.354723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272a100-cda7-434c-8e52-e078908eb700 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:27.377532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:27.398984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3069986-d118-40af-ad1c-d6ef7659f9cd None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:27.440416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a9a54-630f-4dea-b3c9-ca5ef258478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:27.441722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a9a54-630f-4dea-b3c9-ca5ef258478d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:27.442252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a9a54-630f-4dea-b3c9-ca5ef258478d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:27.442735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a9a54-630f-4dea-b3c9-ca5ef258478d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:27.444805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:27.453861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a9a54-630f-4dea-b3c9-ca5ef258478d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:27.455892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a9a54-630f-4dea-b3c9-ca5ef258478d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-5f3a9a54-630f-4dea-b3c9-ca5ef258478d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:27.472570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:27.473522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:27.474411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:27.474411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:27.474411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:27.474411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:27.474411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:27.474411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:27.474411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:27.474411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:27.476541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272a100-cda7-434c-8e52-e078908eb700 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:27.481925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 416/867] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:27.483180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a9a54-630f-4dea-b3c9-ca5ef258478d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:27.517699 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f14842-92d4-4768-bdb6-183e80728c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:27.518556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f14842-92d4-4768-bdb6-183e80728c6b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:27.518931 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f14842-92d4-4768-bdb6-183e80728c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:27.519300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f14842-92d4-4768-bdb6-183e80728c6b None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:27.520311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f14842-92d4-4768-bdb6-183e80728c6b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:27.520947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f14842-92d4-4768-bdb6-183e80728c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:27.522307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f14842-92d4-4768-bdb6-183e80728c6b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:27.535054 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 452/868] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:27 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:27.605578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a9a54-630f-4dea-b3c9-ca5ef258478d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:27.647487 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88fd78-a233-4306-936c-43144985faad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:27.648490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88fd78-a233-4306-936c-43144985faad None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:27.648966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88fd78-a233-4306-936c-43144985faad None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:27.649393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88fd78-a233-4306-936c-43144985faad None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:27.650629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88fd78-a233-4306-936c-43144985faad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:27.651097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88fd78-a233-4306-936c-43144985faad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:27.652796 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88fd78-a233-4306-936c-43144985faad None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:27.664328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:27.672012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 417/869] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:27.674941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f14842-92d4-4768-bdb6-183e80728c6b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:27.799396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f14842-92d4-4768-bdb6-183e80728c6b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:27.841905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef325bc9-caef-4a86-b5d3-afd2ba19ded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:27.842585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef325bc9-caef-4a86-b5d3-afd2ba19ded5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:27.842866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef325bc9-caef-4a86-b5d3-afd2ba19ded5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:27.843145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef325bc9-caef-4a86-b5d3-afd2ba19ded5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:27.844022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef325bc9-caef-4a86-b5d3-afd2ba19ded5 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=70b499a32516486d9a2bed34f7e66b76) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:27.844334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef325bc9-caef-4a86-b5d3-afd2ba19ded5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:27.846339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef325bc9-caef-4a86-b5d3-afd2ba19ded5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:27.853686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 453/870] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:27 2022] GET /identity/v3/role_assignments?user.id=70b499a32516486d9a2bed34f7e66b76&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=03b724e03f704f2ab338b64025a3db6d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:28.958186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb720537-1cb1-48ce-aeb9-75dbeaf71648 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:28.959513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb720537-1cb1-48ce-aeb9-75dbeaf71648 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:28.960359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb720537-1cb1-48ce-aeb9-75dbeaf71648 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:28.963647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 418/871] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:28.968825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a46e99-dff1-4668-be30-3b28b54953ab None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:28.969555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a46e99-dff1-4668-be30-3b28b54953ab None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:28.969555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a46e99-dff1-4668-be30-3b28b54953ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:28.974661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:28.988797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-02a46e99-dff1-4668-be30-3b28b54953ab None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:28.990011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-02a46e99-dff1-4668-be30-3b28b54953ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:29.027985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed525b3-e093-48f9-b3bd-1fc7cc05d63a None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:29.028300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed525b3-e093-48f9-b3bd-1fc7cc05d63a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:29.028564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed525b3-e093-48f9-b3bd-1fc7cc05d63a None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:29.030221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 419/872] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:29.035986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d6c86-a5f0-4b13-8a6f-dcc43e3f4bf3 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:29.036280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d6c86-a5f0-4b13-8a6f-dcc43e3f4bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:29.036548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d6c86-a5f0-4b13-8a6f-dcc43e3f4bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:29.054373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-c58d6c86-a5f0-4b13-8a6f-dcc43e3f4bf3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:29.055614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c58d6c86-a5f0-4b13-8a6f-dcc43e3f4bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:29.079357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 454/873] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:29.103837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a46e99-dff1-4668-be30-3b28b54953ab None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:29.131499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 420/874] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:29.155276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d6c86-a5f0-4b13-8a6f-dcc43e3f4bf3 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:29.216600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a46e99-dff1-4668-be30-3b28b54953ab None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:29.252032 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:29.256715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0916f4-cd6e-4303-9c52-d8f2c3d6b46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:29.257518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0916f4-cd6e-4303-9c52-d8f2c3d6b46f None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:29.257873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0916f4-cd6e-4303-9c52-d8f2c3d6b46f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:29.258218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0916f4-cd6e-4303-9c52-d8f2c3d6b46f None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:29.265528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0916f4-cd6e-4303-9c52-d8f2c3d6b46f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:29.265528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0916f4-cd6e-4303-9c52-d8f2c3d6b46f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:29.265842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d6c86-a5f0-4b13-8a6f-dcc43e3f4bf3 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:29.300788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:29.309783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de49be0-98d4-497a-8a5f-77aa829afa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:29.311373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de49be0-98d4-497a-8a5f-77aa829afa98 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:29.311911 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de49be0-98d4-497a-8a5f-77aa829afa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:29.312423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de49be0-98d4-497a-8a5f-77aa829afa98 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:29.325080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de49be0-98d4-497a-8a5f-77aa829afa98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-6de49be0-98d4-497a-8a5f-77aa829afa98 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:29.333557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:29.334583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:29.335483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 421/875] 10.222.0.64 () {56 vars in 1120 bytes} [Mon Jun 27 09:16:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:29.342089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de49be0-98d4-497a-8a5f-77aa829afa98 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:29.415228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 455/876] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:16:29 2022] POST /identity/v3/services => generated 251 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:16:29.470612 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de49be0-98d4-497a-8a5f-77aa829afa98 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:29.528411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36cddc-a6bf-4768-bfcf-c118c1b6d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:29.529248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36cddc-a6bf-4768-bfcf-c118c1b6d35c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:29.529537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36cddc-a6bf-4768-bfcf-c118c1b6d35c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:29.529814 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36cddc-a6bf-4768-bfcf-c118c1b6d35c None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:29.530690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36cddc-a6bf-4768-bfcf-c118c1b6d35c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:29.530997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36cddc-a6bf-4768-bfcf-c118c1b6d35c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:29.532629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36cddc-a6bf-4768-bfcf-c118c1b6d35c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:29.543820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 422/877] 10.222.0.64 () {56 vars in 1115 bytes} [Mon Jun 27 09:16:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:29.553401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0916f4-cd6e-4303-9c52-d8f2c3d6b46f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:29.675393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0916f4-cd6e-4303-9c52-d8f2c3d6b46f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:29.721827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9df55-a808-438f-a5fb-d7478dc5d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:29.723247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9df55-a808-438f-a5fb-d7478dc5d476 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:29.723859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9df55-a808-438f-a5fb-d7478dc5d476 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:29.724344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9df55-a808-438f-a5fb-d7478dc5d476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:29.746825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9df55-a808-438f-a5fb-d7478dc5d476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:29.750042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9df55-a808-438f-a5fb-d7478dc5d476 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-c3f9df55-a808-438f-a5fb-d7478dc5d476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:29.777777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:29.778935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:29.780065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:29.780065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:29.780065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:29.780065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:29.782204 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 456/878] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:29.788674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36cddc-a6bf-4768-bfcf-c118c1b6d35c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:29.871836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:29.944207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ae209-41db-4c1e-a970-8e88d6112e13 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:29.944485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ae209-41db-4c1e-a970-8e88d6112e13 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:29.944794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ae209-41db-4c1e-a970-8e88d6112e13 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:29.947588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 457/879] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:29.952246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb4ab2-d66a-4de9-a572-4c98163a3300 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:29.952552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb4ab2-d66a-4de9-a572-4c98163a3300 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:29.952811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb4ab2-d66a-4de9-a572-4c98163a3300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:29.975180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-23cb4ab2-d66a-4de9-a572-4c98163a3300 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:29.976618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-23cb4ab2-d66a-4de9-a572-4c98163a3300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:29.977247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36cddc-a6bf-4768-bfcf-c118c1b6d35c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:30.024697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:30.026272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384a3ac-937a-4806-8e8e-c99f2b64593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:30.026965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a3ac-937a-4806-8e8e-c99f2b64593a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:30.027261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a3ac-937a-4806-8e8e-c99f2b64593a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:30.027552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a3ac-937a-4806-8e8e-c99f2b64593a None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:30.028305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384a3ac-937a-4806-8e8e-c99f2b64593a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:30.028618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384a3ac-937a-4806-8e8e-c99f2b64593a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:30.030353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384a3ac-937a-4806-8e8e-c99f2b64593a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:30.044380 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 423/880] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:30.051649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384a3ac-937a-4806-8e8e-c99f2b64593a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:30.063428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 458/881] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:29 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:30.091096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb4ab2-d66a-4de9-a572-4c98163a3300 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:30.237368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384a3ac-937a-4806-8e8e-c99f2b64593a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:30.241983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb4ab2-d66a-4de9-a572-4c98163a3300 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:30.263243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:30.303852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd027b22-817d-42b4-b08d-a54d177b4389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:30.304976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd027b22-817d-42b4-b08d-a54d177b4389 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:30.305698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd027b22-817d-42b4-b08d-a54d177b4389 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:30.306029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd027b22-817d-42b4-b08d-a54d177b4389 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:30.307117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd027b22-817d-42b4-b08d-a54d177b4389 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:30.309082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd027b22-817d-42b4-b08d-a54d177b4389 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:30.324606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229dca9-0108-4ee7-a707-a53043f9e9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:30.325515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229dca9-0108-4ee7-a707-a53043f9e9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:30.325813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229dca9-0108-4ee7-a707-a53043f9e9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:30.326106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229dca9-0108-4ee7-a707-a53043f9e9c5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:30.334451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229dca9-0108-4ee7-a707-a53043f9e9c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:30.335994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229dca9-0108-4ee7-a707-a53043f9e9c5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-b229dca9-0108-4ee7-a707-a53043f9e9c5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:30.353019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:30.354349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:30.356288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:30.359002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 424/882] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:16:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:30.360581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9524689-a7e8-497b-a940-9159d4ceb83b None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:30.365668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9524689-a7e8-497b-a940-9159d4ceb83b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:30.365668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9524689-a7e8-497b-a940-9159d4ceb83b None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:30.365668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 425/883] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:30.365668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9524689-a7e8-497b-a940-9159d4ceb83b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:30.403362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 459/884] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:16:30 2022] POST /identity/v3/services => generated 269 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:16:30.404920 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea2973-9129-488f-9851-9a284e63f5c5 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:30.405366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea2973-9129-488f-9851-9a284e63f5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:30.405646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea2973-9129-488f-9851-9a284e63f5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:30.427522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5aea2973-9129-488f-9851-9a284e63f5c5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:30.429170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5aea2973-9129-488f-9851-9a284e63f5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:30.471469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:30.540443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9524689-a7e8-497b-a940-9159d4ceb83b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:30.546657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 460/885] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:30.565673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea2973-9129-488f-9851-9a284e63f5c5 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:30.589323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bea02-6720-4581-8065-a4129629269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:30.590201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bea02-6720-4581-8065-a4129629269c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:30.590591 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bea02-6720-4581-8065-a4129629269c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:30.590997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bea02-6720-4581-8065-a4129629269c None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:30.592468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bea02-6720-4581-8065-a4129629269c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:30.592468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bea02-6720-4581-8065-a4129629269c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:30.594117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bea02-6720-4581-8065-a4129629269c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:30.620430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 426/886] 10.222.0.64 () {56 vars in 1149 bytes} [Mon Jun 27 09:16:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:30.627061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bea02-6720-4581-8065-a4129629269c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:30.664245 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:30.711494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea2973-9129-488f-9851-9a284e63f5c5 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:30.754468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd097d62-2169-4f0a-84ad-219e82d55db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:30.755764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd097d62-2169-4f0a-84ad-219e82d55db1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:30.756248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd097d62-2169-4f0a-84ad-219e82d55db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:30.756634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd097d62-2169-4f0a-84ad-219e82d55db1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:30.757575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd097d62-2169-4f0a-84ad-219e82d55db1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:30.760165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd097d62-2169-4f0a-84ad-219e82d55db1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-cd097d62-2169-4f0a-84ad-219e82d55db1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:30.781256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:30.783611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:30.785495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:30.790011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 461/887] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:30.791144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd097d62-2169-4f0a-84ad-219e82d55db1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:30.811410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bea02-6720-4581-8065-a4129629269c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:30.842267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:30.862145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdfacc-be88-4a48-a49b-bcb39865ce26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:30.863061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdfacc-be88-4a48-a49b-bcb39865ce26 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:30.863503 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdfacc-be88-4a48-a49b-bcb39865ce26 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:30.863944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdfacc-be88-4a48-a49b-bcb39865ce26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:30.941843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd097d62-2169-4f0a-84ad-219e82d55db1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:30.974620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdfacc-be88-4a48-a49b-bcb39865ce26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:30.979599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdfacc-be88-4a48-a49b-bcb39865ce26 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:30.987044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d23c19-77d1-45bc-8259-b1520f647e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:30.987941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d23c19-77d1-45bc-8259-b1520f647e98 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:30.988325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d23c19-77d1-45bc-8259-b1520f647e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:30.988713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d23c19-77d1-45bc-8259-b1520f647e98 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:30.989557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d23c19-77d1-45bc-8259-b1520f647e98 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:30.989946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d23c19-77d1-45bc-8259-b1520f647e98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:30.992637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d23c19-77d1-45bc-8259-b1520f647e98 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:31.004073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 462/888] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:31.005463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d23c19-77d1-45bc-8259-b1520f647e98 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:31.025103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:31.112082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:31.125265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 427/889] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:30 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:16:31.126278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdfacc-be88-4a48-a49b-bcb39865ce26 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:31.126466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d23c19-77d1-45bc-8259-b1520f647e98 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:31.166856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6210bd-2f67-48cd-8ebc-1c3a89686c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:31.167757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6210bd-2f67-48cd-8ebc-1c3a89686c81 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:31.168124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6210bd-2f67-48cd-8ebc-1c3a89686c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:31.168474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6210bd-2f67-48cd-8ebc-1c3a89686c81 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:31.169272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6210bd-2f67-48cd-8ebc-1c3a89686c81 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:31.169738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6210bd-2f67-48cd-8ebc-1c3a89686c81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:31.171264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6210bd-2f67-48cd-8ebc-1c3a89686c81 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:31.184281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 463/890] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:31.245469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdfacc-be88-4a48-a49b-bcb39865ce26 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:31.295459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:31.300887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439c79e-5f08-45c6-be92-25b85e1cc1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:31.302307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439c79e-5f08-45c6-be92-25b85e1cc1ac None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:31.302805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439c79e-5f08-45c6-be92-25b85e1cc1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:31.303282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439c79e-5f08-45c6-be92-25b85e1cc1ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:31.323500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439c79e-5f08-45c6-be92-25b85e1cc1ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:31.326449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439c79e-5f08-45c6-be92-25b85e1cc1ac None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-c439c79e-5f08-45c6-be92-25b85e1cc1ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:31.362075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:31.363104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:31.364083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:31.364083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:31.364083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:31.364083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:31.364625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 428/891] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:31.373362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6210bd-2f67-48cd-8ebc-1c3a89686c81 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:31.551099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6210bd-2f67-48cd-8ebc-1c3a89686c81 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:31.592073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd6292-635c-465a-a554-8f85fd1b9d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:31.592754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6292-635c-465a-a554-8f85fd1b9d45 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:31.593032 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6292-635c-465a-a554-8f85fd1b9d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:31.593306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6292-635c-465a-a554-8f85fd1b9d45 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:31.594045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd6292-635c-465a-a554-8f85fd1b9d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:31.594348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd6292-635c-465a-a554-8f85fd1b9d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:31.595967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd6292-635c-465a-a554-8f85fd1b9d45 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:31.610523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 464/892] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:31.617120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439c79e-5f08-45c6-be92-25b85e1cc1ac None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:31.735849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439c79e-5f08-45c6-be92-25b85e1cc1ac None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:31.775178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4749ffe-10ce-4d78-9ff0-5422c42c6b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:31.776281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4749ffe-10ce-4d78-9ff0-5422c42c6b71 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:31.776559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4749ffe-10ce-4d78-9ff0-5422c42c6b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:31.776886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4749ffe-10ce-4d78-9ff0-5422c42c6b71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:31.780967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39ad71-622c-470e-b3c7-778fb7f2745e None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:31.781259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39ad71-622c-470e-b3c7-778fb7f2745e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:31.781507 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39ad71-622c-470e-b3c7-778fb7f2745e None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:31.783153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 465/893] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:31.789191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4749ffe-10ce-4d78-9ff0-5422c42c6b71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:31.790798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4749ffe-10ce-4d78-9ff0-5422c42c6b71 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:31.791075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f22c3-d068-4b58-b075-6302a53f72da None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:31.791584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f22c3-d068-4b58-b075-6302a53f72da None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:31.792000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f22c3-d068-4b58-b075-6302a53f72da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-b4749ffe-10ce-4d78-9ff0-5422c42c6b71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:31.809877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:31.813077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:31.815751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:31.815751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:31.815751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:31.815751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:31.815751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:31.815751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:31.815751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:31.815751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:31.815751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 429/894] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:31.818945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4749ffe-10ce-4d78-9ff0-5422c42c6b71 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:31.826300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ed6f22c3-d068-4b58-b075-6302a53f72da None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:31.832937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ed6f22c3-d068-4b58-b075-6302a53f72da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:31.845273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:31.875157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:31.957132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4749ffe-10ce-4d78-9ff0-5422c42c6b71 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:31.965501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 466/895] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:32.010852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f22c3-d068-4b58-b075-6302a53f72da None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:32.022272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afceb9-9b07-491b-8be3-411135bd7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:32.023656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afceb9-9b07-491b-8be3-411135bd7978 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:32.024081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afceb9-9b07-491b-8be3-411135bd7978 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:32.024503 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afceb9-9b07-491b-8be3-411135bd7978 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:32.025708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afceb9-9b07-491b-8be3-411135bd7978 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:32.026208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afceb9-9b07-491b-8be3-411135bd7978 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:32.028562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afceb9-9b07-491b-8be3-411135bd7978 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:32.044055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 430/896] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:32.055566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afceb9-9b07-491b-8be3-411135bd7978 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:32.150106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f22c3-d068-4b58-b075-6302a53f72da None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:32.178677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:32.182946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afceb9-9b07-491b-8be3-411135bd7978 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:32.195466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460d7da-e581-42b0-b4da-202cdb579265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:32.196667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460d7da-e581-42b0-b4da-202cdb579265 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:32.201889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460d7da-e581-42b0-b4da-202cdb579265 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:32.201889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460d7da-e581-42b0-b4da-202cdb579265 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:32.201889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460d7da-e581-42b0-b4da-202cdb579265 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:32.201889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460d7da-e581-42b0-b4da-202cdb579265 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-c460d7da-e581-42b0-b4da-202cdb579265 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:32.213471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:32.214531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 09:16:32.215363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:32.223547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 467/897] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:16:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:32.226574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460d7da-e581-42b0-b4da-202cdb579265 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:32.228851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886bb80-4b82-43e8-8661-5f1e0f99c1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:32.229768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886bb80-4b82-43e8-8661-5f1e0f99c1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:32.230532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886bb80-4b82-43e8-8661-5f1e0f99c1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:32.241303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886bb80-4b82-43e8-8661-5f1e0f99c1f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:32.241303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886bb80-4b82-43e8-8661-5f1e0f99c1f6 None admin] RBAC: Adding query filter params (role.id=e0d6e1d5614e4a4881a0320bda7328cb, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=8731ac7e5a7941f1b2ad04bddc966db6) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:32.241303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886bb80-4b82-43e8-8661-5f1e0f99c1f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:32.241303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886bb80-4b82-43e8-8661-5f1e0f99c1f6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:32.241660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:32.241738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 431/898] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:32 2022] GET /identity/v3/role_assignments?user.id=8731ac7e5a7941f1b2ad04bddc966db6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=e0d6e1d5614e4a4881a0320bda7328cb => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:32.359475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460d7da-e581-42b0-b4da-202cdb579265 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:32.400202 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a92ce3-963a-49a7-8642-58cb67b0bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:32.401523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a92ce3-963a-49a7-8642-58cb67b0bdda None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:32.402020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a92ce3-963a-49a7-8642-58cb67b0bdda None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:32.402519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a92ce3-963a-49a7-8642-58cb67b0bdda None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:32.403517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a92ce3-963a-49a7-8642-58cb67b0bdda None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:32.404115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a92ce3-963a-49a7-8642-58cb67b0bdda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:32.405747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a92ce3-963a-49a7-8642-58cb67b0bdda None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:32.417830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 468/899] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:16:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:32.419764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886bb80-4b82-43e8-8661-5f1e0f99c1f6 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:32.537868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886bb80-4b82-43e8-8661-5f1e0f99c1f6 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:32.582708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61c89-c428-4181-9684-be822b7ea6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:32.583518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61c89-c428-4181-9684-be822b7ea6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:32.583823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61c89-c428-4181-9684-be822b7ea6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:32.584096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61c89-c428-4181-9684-be822b7ea6c3 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:32.584982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb61c89-c428-4181-9684-be822b7ea6c3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:32.585290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb61c89-c428-4181-9684-be822b7ea6c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:32.586939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb61c89-c428-4181-9684-be822b7ea6c3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:32.593873 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 432/900] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:16:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:32.601024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a92ce3-963a-49a7-8642-58cb67b0bdda None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:32.682734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:32.715725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a92ce3-963a-49a7-8642-58cb67b0bdda None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:32.750097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4012a-3557-432c-81ca-358daa93889e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:32.750766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4012a-3557-432c-81ca-358daa93889e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:32.784537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4012a-3557-432c-81ca-358daa93889e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:32.784537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4012a-3557-432c-81ca-358daa93889e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:32.784537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4012a-3557-432c-81ca-358daa93889e None admin] RBAC: Adding query filter params (interface=public, service_id=1f1a3d17017543c3a3ac1b9f6b0c25ef, region_id=RegionOne) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:32.784537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4012a-3557-432c-81ca-358daa93889e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:32.784537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4012a-3557-432c-81ca-358daa93889e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:32.784537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 469/901] 10.222.0.64 () {56 vars in 1251 bytes} [Mon Jun 27 09:16:32 2022] GET /identity/v3/endpoints?service_id=1f1a3d17017543c3a3ac1b9f6b0c25ef&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:32.785334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61c89-c428-4181-9684-be822b7ea6c3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:32.806007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:32.885361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61c89-c428-4181-9684-be822b7ea6c3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:32.923549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e80c38-2e32-43bb-b6e3-34b5018ba081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:32.924340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e80c38-2e32-43bb-b6e3-34b5018ba081 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:32.924621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e80c38-2e32-43bb-b6e3-34b5018ba081 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:32.924934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e80c38-2e32-43bb-b6e3-34b5018ba081 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:32.925856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e80c38-2e32-43bb-b6e3-34b5018ba081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:32.927373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e80c38-2e32-43bb-b6e3-34b5018ba081 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:32.935475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 433/902] 10.222.0.64 () {56 vars in 1087 bytes} [Mon Jun 27 09:16:32 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:16:32.990939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235da32-9316-4d33-b19f-0b2e796618a2 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:32.991272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235da32-9316-4d33-b19f-0b2e796618a2 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:32.991508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235da32-9316-4d33-b19f-0b2e796618a2 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:32.993287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 470/903] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:32.999953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35226d86-4f5e-4190-9676-61b834129b8b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:33.000251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35226d86-4f5e-4190-9676-61b834129b8b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:33.000549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35226d86-4f5e-4190-9676-61b834129b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:33.030703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-35226d86-4f5e-4190-9676-61b834129b8b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:33.033505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-35226d86-4f5e-4190-9676-61b834129b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:33.167605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 434/904] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:33.191750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235da32-9316-4d33-b19f-0b2e796618a2 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:33.339366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235da32-9316-4d33-b19f-0b2e796618a2 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:33.377675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcd565-90ca-4e96-92de-2acf8753b2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:33.378515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcd565-90ca-4e96-92de-2acf8753b2fa None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:33.378840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcd565-90ca-4e96-92de-2acf8753b2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:33.379123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcd565-90ca-4e96-92de-2acf8753b2fa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:33.379829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcd565-90ca-4e96-92de-2acf8753b2fa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:33.381244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcd565-90ca-4e96-92de-2acf8753b2fa None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:33.383463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-e2bcd565-90ca-4e96-92de-2acf8753b2fa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:33.395502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:33.397787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:33.398904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 471/905] 10.222.0.64 () {56 vars in 1132 bytes} [Mon Jun 27 09:16:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:33.403120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35226d86-4f5e-4190-9676-61b834129b8b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:33.428189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:33.524162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35226d86-4f5e-4190-9676-61b834129b8b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:33.557737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df81cf-4f21-4700-8d9f-8ba794756446 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:33.558431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df81cf-4f21-4700-8d9f-8ba794756446 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:33.558431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df81cf-4f21-4700-8d9f-8ba794756446 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:33.560438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 472/906] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:33.561267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3d524-63da-430d-b3c0-2e37823c8e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:33.562105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3d524-63da-430d-b3c0-2e37823c8e07 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:33.562526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3d524-63da-430d-b3c0-2e37823c8e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:33.562938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3d524-63da-430d-b3c0-2e37823c8e07 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:33.563914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3d524-63da-430d-b3c0-2e37823c8e07 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:33.564861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3d524-63da-430d-b3c0-2e37823c8e07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:33.565837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f3236-3ab0-443f-9aa1-bee5f3e75cd7 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:33.566228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f3236-3ab0-443f-9aa1-bee5f3e75cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:33.566644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f3236-3ab0-443f-9aa1-bee5f3e75cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:33.566848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3d524-63da-430d-b3c0-2e37823c8e07 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:33.572849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 435/907] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:33.578481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3d524-63da-430d-b3c0-2e37823c8e07 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:33.587658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ed6f3236-3ab0-443f-9aa1-bee5f3e75cd7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:33.589741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ed6f3236-3ab0-443f-9aa1-bee5f3e75cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:33.683856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 473/908] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:33.688284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:33.705065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3d524-63da-430d-b3c0-2e37823c8e07 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:33.707249 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f3236-3ab0-443f-9aa1-bee5f3e75cd7 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:33.750583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d14757-8435-4195-a43a-ff96d705f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:33.751258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d14757-8435-4195-a43a-ff96d705f618 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:33.751605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d14757-8435-4195-a43a-ff96d705f618 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:33.751884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d14757-8435-4195-a43a-ff96d705f618 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:33.753285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d14757-8435-4195-a43a-ff96d705f618 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:33.753595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d14757-8435-4195-a43a-ff96d705f618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:33.755552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d14757-8435-4195-a43a-ff96d705f618 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:33.763031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 436/909] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:33.768630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d14757-8435-4195-a43a-ff96d705f618 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:33.831234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f3236-3ab0-443f-9aa1-bee5f3e75cd7 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:33.874188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:33.884187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3059f90-6f2a-4daa-9a62-c54d79263478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:33.889395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059f90-6f2a-4daa-9a62-c54d79263478 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:33.889395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059f90-6f2a-4daa-9a62-c54d79263478 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:33.889395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059f90-6f2a-4daa-9a62-c54d79263478 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:33.889395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3059f90-6f2a-4daa-9a62-c54d79263478 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:33.889395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3059f90-6f2a-4daa-9a62-c54d79263478 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:33.927293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d14757-8435-4195-a43a-ff96d705f618 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:33.963105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 474/910] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:16:33 2022] POST /identity/v3/services => generated 241 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:16:33.991043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f50c64-6a38-426a-becc-3fff7c334cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:33.993109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50c64-6a38-426a-becc-3fff7c334cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:33.994017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50c64-6a38-426a-becc-3fff7c334cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:33.994802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50c64-6a38-426a-becc-3fff7c334cd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:33.996790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50c64-6a38-426a-becc-3fff7c334cd1 None admin] RBAC: Adding query filter params (interface=public, service_id=925ec61f37bf4ae4a4387a6785ab56fb, region_id=RegionOne) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:33.997719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50c64-6a38-426a-becc-3fff7c334cd1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:34.001205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50c64-6a38-426a-becc-3fff7c334cd1 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:34.019037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 437/911] 10.222.0.64 () {56 vars in 1251 bytes} [Mon Jun 27 09:16:33 2022] GET /identity/v3/endpoints?service_id=925ec61f37bf4ae4a4387a6785ab56fb&interface=public®ion_id=RegionOne => generated 194 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:34.024267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3059f90-6f2a-4daa-9a62-c54d79263478 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:34.202388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3059f90-6f2a-4daa-9a62-c54d79263478 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:34.237246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8fb99-d076-4ddf-9359-a97bdfdb3140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:34.237928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8fb99-d076-4ddf-9359-a97bdfdb3140 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:34.238213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8fb99-d076-4ddf-9359-a97bdfdb3140 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:34.238490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8fb99-d076-4ddf-9359-a97bdfdb3140 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:34.239176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8fb99-d076-4ddf-9359-a97bdfdb3140 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:34.240608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8fb99-d076-4ddf-9359-a97bdfdb3140 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:34.251762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 475/912] 10.222.0.64 () {56 vars in 1087 bytes} [Mon Jun 27 09:16:34 2022] GET /identity/v3/services => generated 1497 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:16:34.487511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279aa8b-7362-4f9b-9ffb-365f6139eea1 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:34.488083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279aa8b-7362-4f9b-9ffb-365f6139eea1 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:34.488083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279aa8b-7362-4f9b-9ffb-365f6139eea1 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:34.489924 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 438/913] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:34.495726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae49db-5abb-40b6-8dce-64265e42dfae None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:34.496014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae49db-5abb-40b6-8dce-64265e42dfae None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:34.496298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae49db-5abb-40b6-8dce-64265e42dfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:34.515914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-76ae49db-5abb-40b6-8dce-64265e42dfae None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:34.518024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-76ae49db-5abb-40b6-8dce-64265e42dfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:34.569144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:34.590854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 476/914] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:34.615691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279aa8b-7362-4f9b-9ffb-365f6139eea1 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:34.630282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:34.762026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279aa8b-7362-4f9b-9ffb-365f6139eea1 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:34.806077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab439b-1861-48b9-932e-8738c2c85395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:34.807299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab439b-1861-48b9-932e-8738c2c85395 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:34.807890 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab439b-1861-48b9-932e-8738c2c85395 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:34.808399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab439b-1861-48b9-932e-8738c2c85395 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:34.821991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab439b-1861-48b9-932e-8738c2c85395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-82ab439b-1861-48b9-932e-8738c2c85395 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:34.831057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:34.832322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:34.833163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:34.839672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 439/915] 10.222.0.64 () {56 vars in 1120 bytes} [Mon Jun 27 09:16:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:34.842714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae49db-5abb-40b6-8dce-64265e42dfae None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:34.965829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae49db-5abb-40b6-8dce-64265e42dfae None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:35.007326 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc051b-c711-4f26-830a-91758677154d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:35.009217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc051b-c711-4f26-830a-91758677154d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:35.009217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc051b-c711-4f26-830a-91758677154d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:35.009436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc051b-c711-4f26-830a-91758677154d None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:35.010872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc051b-c711-4f26-830a-91758677154d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:35.011432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc051b-c711-4f26-830a-91758677154d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:35.013660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc051b-c711-4f26-830a-91758677154d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:35.025727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 477/916] 10.222.0.64 () {56 vars in 1115 bytes} [Mon Jun 27 09:16:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:35.031807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab439b-1861-48b9-932e-8738c2c85395 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:35.134819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:35.172487 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab439b-1861-48b9-932e-8738c2c85395 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:35.181411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc217d4-bb65-486e-81f1-7aab7e4778b1 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:35.181722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc217d4-bb65-486e-81f1-7aab7e4778b1 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:35.181977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc217d4-bb65-486e-81f1-7aab7e4778b1 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:35.183782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 478/917] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:35.189959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa4793-0ab5-4b79-b3be-116e4471384f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:35.190336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa4793-0ab5-4b79-b3be-116e4471384f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:35.190561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa4793-0ab5-4b79-b3be-116e4471384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:35.210855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-33fa4793-0ab5-4b79-b3be-116e4471384f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:35.212242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-33fa4793-0ab5-4b79-b3be-116e4471384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:35.225865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5814a1-c572-449e-8925-0df7847f4131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:35.227020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5814a1-c572-449e-8925-0df7847f4131 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:35.230340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5814a1-c572-449e-8925-0df7847f4131 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:35.230340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5814a1-c572-449e-8925-0df7847f4131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:35.240930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5814a1-c572-449e-8925-0df7847f4131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:35.243769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5814a1-c572-449e-8925-0df7847f4131 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:35.259973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-6f5814a1-c572-449e-8925-0df7847f4131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:35.289834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:35.290887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:35.291937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:35.291937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:35.291937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:35.291937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:35.299673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 440/918] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:35.301649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5814a1-c572-449e-8925-0df7847f4131 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:35.319930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 479/919] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:35.360727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa4793-0ab5-4b79-b3be-116e4471384f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:35.431925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5814a1-c572-449e-8925-0df7847f4131 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:35.457209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:35.475944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425d78c-1260-4a12-888b-e503903e49be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:35.476638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425d78c-1260-4a12-888b-e503903e49be None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:35.476922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425d78c-1260-4a12-888b-e503903e49be None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:35.477199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425d78c-1260-4a12-888b-e503903e49be None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:35.477944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425d78c-1260-4a12-888b-e503903e49be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:35.478258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425d78c-1260-4a12-888b-e503903e49be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:35.479790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425d78c-1260-4a12-888b-e503903e49be None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:35.497626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 441/920] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:35.503133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425d78c-1260-4a12-888b-e503903e49be None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:35.517751 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa4793-0ab5-4b79-b3be-116e4471384f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:35.598800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d8cc6-9c7c-4571-b676-63f73860e594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:35.600530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d8cc6-9c7c-4571-b676-63f73860e594 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:35.601864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d8cc6-9c7c-4571-b676-63f73860e594 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:35.602374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d8cc6-9c7c-4571-b676-63f73860e594 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:35.604029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d8cc6-9c7c-4571-b676-63f73860e594 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:35.608197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d8cc6-9c7c-4571-b676-63f73860e594 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-2e1d8cc6-9c7c-4571-b676-63f73860e594 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:35.640400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:35.642637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 09:16:35.644111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:35.651574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 480/921] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:16:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:35.656236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d8cc6-9c7c-4571-b676-63f73860e594 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:35.658938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425d78c-1260-4a12-888b-e503903e49be None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:35.697517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3177a48-ef1b-4167-8d3e-906d93b8e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:35.698816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3177a48-ef1b-4167-8d3e-906d93b8e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:35.698816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3177a48-ef1b-4167-8d3e-906d93b8e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:35.699107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3177a48-ef1b-4167-8d3e-906d93b8e8d0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:35.707376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3177a48-ef1b-4167-8d3e-906d93b8e8d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:35.708882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3177a48-ef1b-4167-8d3e-906d93b8e8d0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-c3177a48-ef1b-4167-8d3e-906d93b8e8d0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:35.725158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:35.727078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:35.729077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:35.731273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 442/922] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:16:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:35.732619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3177a48-ef1b-4167-8d3e-906d93b8e8d0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:35.745640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:35.779193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:35.844877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3177a48-ef1b-4167-8d3e-906d93b8e8d0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:35.881128 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d8cc6-9c7c-4571-b676-63f73860e594 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:35.885059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625c59-aa2f-4fb9-a36d-23c0cf86dfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:35.885922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625c59-aa2f-4fb9-a36d-23c0cf86dfff None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:35.886314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625c59-aa2f-4fb9-a36d-23c0cf86dfff None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:35.886696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625c59-aa2f-4fb9-a36d-23c0cf86dfff None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:35.887946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625c59-aa2f-4fb9-a36d-23c0cf86dfff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:35.889496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625c59-aa2f-4fb9-a36d-23c0cf86dfff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:35.890007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625c59-aa2f-4fb9-a36d-23c0cf86dfff None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:35.915798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 443/923] 10.222.0.64 () {56 vars in 1149 bytes} [Mon Jun 27 09:16:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:35.917312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625c59-aa2f-4fb9-a36d-23c0cf86dfff None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:35.957146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3bf80-9b40-4991-b6a1-452904a7bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:35.958697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3bf80-9b40-4991-b6a1-452904a7bef2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:35.959428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3bf80-9b40-4991-b6a1-452904a7bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:35.960279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3bf80-9b40-4991-b6a1-452904a7bef2 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:35.961831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3bf80-9b40-4991-b6a1-452904a7bef2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:35.962827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3bf80-9b40-4991-b6a1-452904a7bef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:35.965953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3bf80-9b40-4991-b6a1-452904a7bef2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:35.983915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 481/924] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:16:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:35.988852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3bf80-9b40-4991-b6a1-452904a7bef2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:36.027740 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:36.036855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625c59-aa2f-4fb9-a36d-23c0cf86dfff None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:36.102140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac35ea2-0f1c-4fc5-9f8b-bc9c5cc626e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:36.103286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac35ea2-0f1c-4fc5-9f8b-bc9c5cc626e3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.103830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac35ea2-0f1c-4fc5-9f8b-bc9c5cc626e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.104293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac35ea2-0f1c-4fc5-9f8b-bc9c5cc626e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.163551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac35ea2-0f1c-4fc5-9f8b-bc9c5cc626e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:36.163551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac35ea2-0f1c-4fc5-9f8b-bc9c5cc626e3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:36.163854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3bf80-9b40-4991-b6a1-452904a7bef2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-2ac35ea2-0f1c-4fc5-9f8b-bc9c5cc626e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:36.164041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:36.165665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:36.167471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:36.167471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:36.167471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:36.167471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:36.171901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:36.172144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 444/925] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:36.176052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac35ea2-0f1c-4fc5-9f8b-bc9c5cc626e3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:36.238392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1df22-61a8-47d0-b28c-271aa7dbdc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:36.239177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1df22-61a8-47d0-b28c-271aa7dbdc35 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.239310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1df22-61a8-47d0-b28c-271aa7dbdc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.239731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1df22-61a8-47d0-b28c-271aa7dbdc35 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.240380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1df22-61a8-47d0-b28c-271aa7dbdc35 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:36.240693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1df22-61a8-47d0-b28c-271aa7dbdc35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:36.242111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1df22-61a8-47d0-b28c-271aa7dbdc35 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:36.253344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 482/926] 10.222.0.64 () {56 vars in 1114 bytes} [Mon Jun 27 09:16:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:36.256832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1df22-61a8-47d0-b28c-271aa7dbdc35 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:36.376467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac35ea2-0f1c-4fc5-9f8b-bc9c5cc626e3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:36.388040 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1df22-61a8-47d0-b28c-271aa7dbdc35 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:36.429359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c51d6e-dab4-43c7-b40e-de82cfd8f439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:36.429966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c51d6e-dab4-43c7-b40e-de82cfd8f439 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.430250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c51d6e-dab4-43c7-b40e-de82cfd8f439 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.430519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c51d6e-dab4-43c7-b40e-de82cfd8f439 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.430624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c499eb5-e69d-4d44-8479-db40e849496b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:36.431275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c51d6e-dab4-43c7-b40e-de82cfd8f439 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:36.431382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c499eb5-e69d-4d44-8479-db40e849496b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.431665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c499eb5-e69d-4d44-8479-db40e849496b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.431939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c499eb5-e69d-4d44-8479-db40e849496b None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.432744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c51d6e-dab4-43c7-b40e-de82cfd8f439 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:36.433041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c499eb5-e69d-4d44-8479-db40e849496b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:36.433285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c499eb5-e69d-4d44-8479-db40e849496b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:36.434910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c499eb5-e69d-4d44-8479-db40e849496b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:36.448394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 445/927] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:36.449854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabaea7-cf37-4f34-b73a-c9bf9b534704 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.450573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabaea7-cf37-4f34-b73a-c9bf9b534704 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.457448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabaea7-cf37-4f34-b73a-c9bf9b534704 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.457448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 446/928] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:36.457448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4803-63d8-4ba3-8a1b-fbd9ac405f45 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.457448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4803-63d8-4ba3-8a1b-fbd9ac405f45 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.457448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4803-63d8-4ba3-8a1b-fbd9ac405f45 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.457448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 447/929] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:36.457448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4803-63d8-4ba3-8a1b-fbd9ac405f45 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:36.472024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:36.525040 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 483/930] 10.222.0.64 () {60 vars in 1144 bytes} [Mon Jun 27 09:16:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:16:36.526791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5380752-630c-4d02-a0bb-08eef609f133 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.527173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5380752-630c-4d02-a0bb-08eef609f133 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.527596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5380752-630c-4d02-a0bb-08eef609f133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.551182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c5380752-630c-4d02-a0bb-08eef609f133 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:36.552545 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c5380752-630c-4d02-a0bb-08eef609f133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:36.573386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:36.590173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4803-63d8-4ba3-8a1b-fbd9ac405f45 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:36.648783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fda0d2-d7f2-4da0-99b5-59820445ca0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:36.649610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fda0d2-d7f2-4da0-99b5-59820445ca0c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.649884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fda0d2-d7f2-4da0-99b5-59820445ca0c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.650156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fda0d2-d7f2-4da0-99b5-59820445ca0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.659888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 484/931] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:36.659888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b9511-c73d-47e4-aab4-5e04623cd31a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.659888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b9511-c73d-47e4-aab4-5e04623cd31a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.659888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b9511-c73d-47e4-aab4-5e04623cd31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.667512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fda0d2-d7f2-4da0-99b5-59820445ca0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:36.669068 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fda0d2-d7f2-4da0-99b5-59820445ca0c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:36.673984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ca9b9511-c73d-47e4-aab4-5e04623cd31a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:36.675354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ca9b9511-c73d-47e4-aab4-5e04623cd31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-71fda0d2-d7f2-4da0-99b5-59820445ca0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:36.684623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:36.685572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:36.686486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:36.686486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:36.686486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:36.686486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:36.686486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:36.686486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:36.686486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:36.686486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:36.686486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 448/932] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:36.687427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fda0d2-d7f2-4da0-99b5-59820445ca0c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:36.783553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 485/933] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:36.790763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:36.790927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b9511-c73d-47e4-aab4-5e04623cd31a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:36.832098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fda0d2-d7f2-4da0-99b5-59820445ca0c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:36.870300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670cb45-9220-4d9d-b52f-991665314cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:36.871114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670cb45-9220-4d9d-b52f-991665314cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.871392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670cb45-9220-4d9d-b52f-991665314cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.871695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670cb45-9220-4d9d-b52f-991665314cc6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.872354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670cb45-9220-4d9d-b52f-991665314cc6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:36.873837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670cb45-9220-4d9d-b52f-991665314cc6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0670cb45-9220-4d9d-b52f-991665314cc6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:36.887929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:36.896334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:36.897883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 449/934] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:36.899381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670cb45-9220-4d9d-b52f-991665314cc6 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:36.916236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:36.940409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b9511-c73d-47e4-aab4-5e04623cd31a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:36.993524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297d689-e18a-41fb-8d75-6a8fe7a63145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:36.994406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297d689-e18a-41fb-8d75-6a8fe7a63145 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:36.994842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297d689-e18a-41fb-8d75-6a8fe7a63145 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:36.995251 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297d689-e18a-41fb-8d75-6a8fe7a63145 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:36.996263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297d689-e18a-41fb-8d75-6a8fe7a63145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:36.996740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297d689-e18a-41fb-8d75-6a8fe7a63145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:36.998362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297d689-e18a-41fb-8d75-6a8fe7a63145 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:37.018262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 486/935] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:37.018956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297d689-e18a-41fb-8d75-6a8fe7a63145 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:37.050421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670cb45-9220-4d9d-b52f-991665314cc6 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:37.083463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:37.091376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b2a27-bb66-4220-be5f-f317c0f1996b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:37.099753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b2a27-bb66-4220-be5f-f317c0f1996b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:37.099753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b2a27-bb66-4220-be5f-f317c0f1996b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:37.099753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b2a27-bb66-4220-be5f-f317c0f1996b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:37.099753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b2a27-bb66-4220-be5f-f317c0f1996b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:37.099753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b2a27-bb66-4220-be5f-f317c0f1996b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-611b2a27-bb66-4220-be5f-f317c0f1996b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:37.109715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:37.111043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 09:16:37.112187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:37.113480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 450/936] 10.222.0.64 () {56 vars in 1132 bytes} [Mon Jun 27 09:16:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:37.113480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b2a27-bb66-4220-be5f-f317c0f1996b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:37.152597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297d689-e18a-41fb-8d75-6a8fe7a63145 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:37.202559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd7e8d-816d-44f5-a4e7-7e9f5b122b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:37.202559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd7e8d-816d-44f5-a4e7-7e9f5b122b03 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:37.202559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd7e8d-816d-44f5-a4e7-7e9f5b122b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:37.202559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd7e8d-816d-44f5-a4e7-7e9f5b122b03 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:37.203034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd7e8d-816d-44f5-a4e7-7e9f5b122b03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:37.203034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd7e8d-816d-44f5-a4e7-7e9f5b122b03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:37.203034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd7e8d-816d-44f5-a4e7-7e9f5b122b03 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:37.208413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 487/937] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:37.213939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd7e8d-816d-44f5-a4e7-7e9f5b122b03 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:37.224547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:37.240515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b2a27-bb66-4220-be5f-f317c0f1996b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:37.295482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa475b7-ee1b-4f0b-8824-c9ad77c48cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:37.296443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa475b7-ee1b-4f0b-8824-c9ad77c48cfe None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:37.296727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa475b7-ee1b-4f0b-8824-c9ad77c48cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:37.297004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa475b7-ee1b-4f0b-8824-c9ad77c48cfe None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/8731ac7e5a7941f1b2ad04bddc966db6/roles/e0d6e1d5614e4a4881a0320bda7328cb` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:37.299455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa475b7-ee1b-4f0b-8824-c9ad77c48cfe None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=8731ac7e5a7941f1b2ad04bddc966db6, role_id=e0d6e1d5614e4a4881a0320bda7328cb)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:37.300967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa475b7-ee1b-4f0b-8824-c9ad77c48cfe None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:37.338322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd7e8d-816d-44f5-a4e7-7e9f5b122b03 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:37.374155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01669186-5ab5-4f04-8a1c-ab3be859fc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:37.374827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01669186-5ab5-4f04-8a1c-ab3be859fc96 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:37.375106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01669186-5ab5-4f04-8a1c-ab3be859fc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:37.375376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01669186-5ab5-4f04-8a1c-ab3be859fc96 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:37.376126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01669186-5ab5-4f04-8a1c-ab3be859fc96 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:37.376458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01669186-5ab5-4f04-8a1c-ab3be859fc96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:37.377832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01669186-5ab5-4f04-8a1c-ab3be859fc96 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:37.390241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 488/938] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:37.390241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01669186-5ab5-4f04-8a1c-ab3be859fc96 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:37.395644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 451/939] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:16:37 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/8731ac7e5a7941f1b2ad04bddc966db6/roles/e0d6e1d5614e4a4881a0320bda7328cb => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:16:37.399721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa475b7-ee1b-4f0b-8824-c9ad77c48cfe None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:37.491765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:37.493679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:37.535083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01669186-5ab5-4f04-8a1c-ab3be859fc96 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:37.550494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa475b7-ee1b-4f0b-8824-c9ad77c48cfe None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:37.578663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3ab16-8f01-4b98-8cb3-a2417f2c0025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:37.580936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3ab16-8f01-4b98-8cb3-a2417f2c0025 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:37.581510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3ab16-8f01-4b98-8cb3-a2417f2c0025 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:37.581978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3ab16-8f01-4b98-8cb3-a2417f2c0025 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:37.583155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3ab16-8f01-4b98-8cb3-a2417f2c0025 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:37.583686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3ab16-8f01-4b98-8cb3-a2417f2c0025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:37.585525 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3ab16-8f01-4b98-8cb3-a2417f2c0025 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:37.594983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 489/940] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:37.601725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3ab16-8f01-4b98-8cb3-a2417f2c0025 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:37.602937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b1d91-081e-4a95-b478-6592cf7214b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:37.602937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b1d91-081e-4a95-b478-6592cf7214b2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:37.603442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b1d91-081e-4a95-b478-6592cf7214b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:37.603442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b1d91-081e-4a95-b478-6592cf7214b2 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:37.604512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b1d91-081e-4a95-b478-6592cf7214b2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:37.604819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b1d91-081e-4a95-b478-6592cf7214b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:37.606483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b1d91-081e-4a95-b478-6592cf7214b2 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:37.617663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 452/941] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:37.624119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b1d91-081e-4a95-b478-6592cf7214b2 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:37.733517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3ab16-8f01-4b98-8cb3-a2417f2c0025 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:37.774807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b1d91-081e-4a95-b478-6592cf7214b2 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:37.779490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09afa9-ea63-440f-bf36-e970e30a8142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:37.780216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09afa9-ea63-440f-bf36-e970e30a8142 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:37.780502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09afa9-ea63-440f-bf36-e970e30a8142 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:37.780783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09afa9-ea63-440f-bf36-e970e30a8142 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:37.781614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09afa9-ea63-440f-bf36-e970e30a8142 None admin] RBAC: Adding query filter params (interface=public, service_id=a3856a8940324d1f979dd592472b1b5b, region_id=RegionOne) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:37.781932 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09afa9-ea63-440f-bf36-e970e30a8142 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:37.783412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09afa9-ea63-440f-bf36-e970e30a8142 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:37.785297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:37.791343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 490/942] 10.222.0.64 () {56 vars in 1251 bytes} [Mon Jun 27 09:16:37 2022] GET /identity/v3/endpoints?service_id=a3856a8940324d1f979dd592472b1b5b&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:37.801064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09afa9-ea63-440f-bf36-e970e30a8142 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:37.801669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:37.818202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152cfb3-0015-460d-abf1-f14ea751a709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:37.819446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152cfb3-0015-460d-abf1-f14ea751a709 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:37.820000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152cfb3-0015-460d-abf1-f14ea751a709 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:37.820389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152cfb3-0015-460d-abf1-f14ea751a709 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:37.821263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152cfb3-0015-460d-abf1-f14ea751a709 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:37.822756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152cfb3-0015-460d-abf1-f14ea751a709 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:37.954460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09afa9-ea63-440f-bf36-e970e30a8142 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:37.981240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 453/943] 10.222.0.64 () {60 vars in 1144 bytes} [Mon Jun 27 09:16:37 2022] POST /identity/v3/endpoints => generated 347 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:16:38.015006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d05daa-35d6-4701-905c-de303af721c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:38.015927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d05daa-35d6-4701-905c-de303af721c9 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:38.016312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d05daa-35d6-4701-905c-de303af721c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:38.016691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d05daa-35d6-4701-905c-de303af721c9 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:38.017563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d05daa-35d6-4701-905c-de303af721c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:38.019198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d05daa-35d6-4701-905c-de303af721c9 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:38.027490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 491/944] 10.222.0.64 () {56 vars in 1087 bytes} [Mon Jun 27 09:16:37 2022] GET /identity/v3/services => generated 1497 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:16:38.741424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d9ea8-a5b7-41b5-912a-d556b9fe5950 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:38.742072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d9ea8-a5b7-41b5-912a-d556b9fe5950 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:38.742072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d9ea8-a5b7-41b5-912a-d556b9fe5950 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:38.743635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 454/945] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:38.749206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e441fe-09cc-4d44-a85a-802c672f854c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:38.749672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e441fe-09cc-4d44-a85a-802c672f854c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:38.749757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e441fe-09cc-4d44-a85a-802c672f854c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:38.767304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d8e441fe-09cc-4d44-a85a-802c672f854c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:38.768534 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d8e441fe-09cc-4d44-a85a-802c672f854c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:38.830868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:38.846160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 492/946] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:38.869168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d9ea8-a5b7-41b5-912a-d556b9fe5950 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:38.988443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d9ea8-a5b7-41b5-912a-d556b9fe5950 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:39.025424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:39.026278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:39.026566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:39.026847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:39.027609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:39.029109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:39.030647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING keystone.server.flask.common [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 09:16:39.084631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.sql.core [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 09:16:39.084631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 09:16:39.084631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [parameters: {'id': 'ba8584ff517c42cb96892143cffaea27', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 09:16:39.084631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80344) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ba8584ff517c42cb96892143cffaea27.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ba8584ff517c42cb96892143cffaea27. Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application conn.query(q) Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 09:16:39.102004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result.read() Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 27 09:16:39.102912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application next(self.gen) Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application yield resource Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application next(self.gen) Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise self.value Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application session.commit() Jun 27 09:16:39.103954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.session.flush() Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application compat.raise_( Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 09:16:39.104869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 09:16:39.107386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application conn.query(q) Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result.read() Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 09:16:39.115846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [parameters: {'id': 'ba8584ff517c42cb96892143cffaea27', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:39.118036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:39.119823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ba8584ff517c42cb96892143cffaea27. Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:39.120987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 455/947] 10.222.0.64 () {60 vars in 1140 bytes} [Mon Jun 27 09:16:38 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:39.121827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e441fe-09cc-4d44-a85a-802c672f854c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:39.244623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e441fe-09cc-4d44-a85a-802c672f854c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:39.293092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44317ac3-504a-4c9d-b6e4-884992273240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:39.293868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44317ac3-504a-4c9d-b6e4-884992273240 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:39.294155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44317ac3-504a-4c9d-b6e4-884992273240 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:39.294433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44317ac3-504a-4c9d-b6e4-884992273240 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:39.304421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44317ac3-504a-4c9d-b6e4-884992273240 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:39.306419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44317ac3-504a-4c9d-b6e4-884992273240 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-44317ac3-504a-4c9d-b6e4-884992273240 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:39.330646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:39.333005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:39.335271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:39.335271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:39.335271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:39.335271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:39.335271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:39.335271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:39.335271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:39.335271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:39.335271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 493/948] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:39.339595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:39.376296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:39.453879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05312048-3da4-4917-9095-014901c5ba2f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:39.497350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bd836-fad3-48c2-8056-effbe87a4d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:39.498361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bd836-fad3-48c2-8056-effbe87a4d7d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:39.498809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bd836-fad3-48c2-8056-effbe87a4d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:39.499243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bd836-fad3-48c2-8056-effbe87a4d7d None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:39.500216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bd836-fad3-48c2-8056-effbe87a4d7d None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:39.500823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bd836-fad3-48c2-8056-effbe87a4d7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:39.502700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bd836-fad3-48c2-8056-effbe87a4d7d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:39.526482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 456/949] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:39.705238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48e6c6-356f-4b52-8e11-24c0acfdf3e7 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:39.705430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48e6c6-356f-4b52-8e11-24c0acfdf3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:39.705708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48e6c6-356f-4b52-8e11-24c0acfdf3e7 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:39.707117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 494/950] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:39.715037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08093-7ca4-4ba5-995b-b5035007a2a0 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:39.715926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08093-7ca4-4ba5-995b-b5035007a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:39.716579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08093-7ca4-4ba5-995b-b5035007a2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:39.748674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d5e08093-7ca4-4ba5-995b-b5035007a2a0 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:39.751638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d5e08093-7ca4-4ba5-995b-b5035007a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:39.851554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 457/951] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:39.870746 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48e6c6-356f-4b52-8e11-24c0acfdf3e7 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:39.954025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:40.004672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48e6c6-356f-4b52-8e11-24c0acfdf3e7 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:40.043695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f952c-db22-425c-8816-a7f02ec5eec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:40.045159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f952c-db22-425c-8816-a7f02ec5eec3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.045709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f952c-db22-425c-8816-a7f02ec5eec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.046224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f952c-db22-425c-8816-a7f02ec5eec3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.059263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f952c-db22-425c-8816-a7f02ec5eec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-806f952c-db22-425c-8816-a7f02ec5eec3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:40.069012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:40.070112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:40.071048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:40.078689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 495/952] 10.222.0.64 () {56 vars in 1120 bytes} [Mon Jun 27 09:16:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:40.084657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08093-7ca4-4ba5-995b-b5035007a2a0 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:40.119116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:40.206390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6de32-c769-490a-bc7e-710f74029a11 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.207068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6de32-c769-490a-bc7e-710f74029a11 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.207627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6de32-c769-490a-bc7e-710f74029a11 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.210436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 496/953] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:40.216125 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8c79-5c20-4fbf-b2b8-7eece6564b29 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.216448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8c79-5c20-4fbf-b2b8-7eece6564b29 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.216695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b8c79-5c20-4fbf-b2b8-7eece6564b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.225992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08093-7ca4-4ba5-995b-b5035007a2a0 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:40.235042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7a2b8c79-5c20-4fbf-b2b8-7eece6564b29 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:40.236826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7a2b8c79-5c20-4fbf-b2b8-7eece6564b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:40.273795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbed1bc-7265-41e9-94c1-2768d627943a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:40.274839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbed1bc-7265-41e9-94c1-2768d627943a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.276011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbed1bc-7265-41e9-94c1-2768d627943a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.276011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbed1bc-7265-41e9-94c1-2768d627943a None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.276974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbed1bc-7265-41e9-94c1-2768d627943a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:40.277443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbed1bc-7265-41e9-94c1-2768d627943a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:40.279799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbed1bc-7265-41e9-94c1-2768d627943a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:40.295621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 458/954] 10.222.0.64 () {56 vars in 1115 bytes} [Mon Jun 27 09:16:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:40.297861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbed1bc-7265-41e9-94c1-2768d627943a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:40.335190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 497/955] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:40.358432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b8c79-5c20-4fbf-b2b8-7eece6564b29 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:40.404930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:40.409331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:40.426157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbed1bc-7265-41e9-94c1-2768d627943a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:40.466650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412bd77-94ef-40e3-9ad6-088d370c0c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:40.467459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412bd77-94ef-40e3-9ad6-088d370c0c0f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.467770 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412bd77-94ef-40e3-9ad6-088d370c0c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.468048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412bd77-94ef-40e3-9ad6-088d370c0c0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.480198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412bd77-94ef-40e3-9ad6-088d370c0c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:40.481694 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412bd77-94ef-40e3-9ad6-088d370c0c0f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:40.485273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b8c79-5c20-4fbf-b2b8-7eece6564b29 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-9412bd77-94ef-40e3-9ad6-088d370c0c0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:40.506151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:40.508928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:40.510707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:40.510707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:40.510707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:40.510707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:40.510707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 459/956] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:40.510707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8106b67-82e2-43f8-b042-270049213937 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.510707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8106b67-82e2-43f8-b042-270049213937 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.510707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8106b67-82e2-43f8-b042-270049213937 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.512586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 460/957] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:40.516908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8106b67-82e2-43f8-b042-270049213937 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:40.542832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6efa8-cb2b-4b4c-99a5-3350a7b71e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:40.543582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6efa8-cb2b-4b4c-99a5-3350a7b71e5a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.543899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6efa8-cb2b-4b4c-99a5-3350a7b71e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.544167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6efa8-cb2b-4b4c-99a5-3350a7b71e5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.544888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6efa8-cb2b-4b4c-99a5-3350a7b71e5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:40.546441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6efa8-cb2b-4b4c-99a5-3350a7b71e5a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-b2f6efa8-cb2b-4b4c-99a5-3350a7b71e5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:40.561074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:40.562219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:40.563086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 498/958] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:40.564006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce7e76-61e5-4070-aa85-206978ec9bd8 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.564006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce7e76-61e5-4070-aa85-206978ec9bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.564139 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce7e76-61e5-4070-aa85-206978ec9bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.581317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a1ce7e76-61e5-4070-aa85-206978ec9bd8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:40.582521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a1ce7e76-61e5-4070-aa85-206978ec9bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:40.661356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 499/959] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:40.662211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce7e76-61e5-4070-aa85-206978ec9bd8 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:40.677504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8106b67-82e2-43f8-b042-270049213937 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:40.694460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:40.722069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb1a98-3843-40d5-9728-559f07f5d3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:40.723622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb1a98-3843-40d5-9728-559f07f5d3fc None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.723622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb1a98-3843-40d5-9728-559f07f5d3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.723622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb1a98-3843-40d5-9728-559f07f5d3fc None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.724165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb1a98-3843-40d5-9728-559f07f5d3fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:40.724476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb1a98-3843-40d5-9728-559f07f5d3fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:40.725927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb1a98-3843-40d5-9728-559f07f5d3fc None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:40.727985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:40.747713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 461/960] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:40.747713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb1a98-3843-40d5-9728-559f07f5d3fc None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:40.805713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce7e76-61e5-4070-aa85-206978ec9bd8 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:40.847451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70a7ec-c05d-42bb-b10f-5c33603ba664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:40.848484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70a7ec-c05d-42bb-b10f-5c33603ba664 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.848976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70a7ec-c05d-42bb-b10f-5c33603ba664 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.849306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70a7ec-c05d-42bb-b10f-5c33603ba664 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.849976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70a7ec-c05d-42bb-b10f-5c33603ba664 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:40.850288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70a7ec-c05d-42bb-b10f-5c33603ba664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:40.851715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70a7ec-c05d-42bb-b10f-5c33603ba664 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:40.857392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 500/961] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:40.858318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70a7ec-c05d-42bb-b10f-5c33603ba664 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:40.888944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb1a98-3843-40d5-9728-559f07f5d3fc None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:40.928579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f47af3-a6dd-4b42-b931-b655b9be0cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:40.928579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f47af3-a6dd-4b42-b931-b655b9be0cee None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:40.928579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f47af3-a6dd-4b42-b931-b655b9be0cee None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:40.928579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f47af3-a6dd-4b42-b931-b655b9be0cee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:40.929226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f47af3-a6dd-4b42-b931-b655b9be0cee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:40.930475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f47af3-a6dd-4b42-b931-b655b9be0cee None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-d3f47af3-a6dd-4b42-b931-b655b9be0cee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:40.943132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:40.944203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:40.945695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 462/962] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:40.946489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f47af3-a6dd-4b42-b931-b655b9be0cee None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:40.976518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:40.982566 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70a7ec-c05d-42bb-b10f-5c33603ba664 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:41.021440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7fdfa-bb1e-489d-8502-6329d5962f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:41.022277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7fdfa-bb1e-489d-8502-6329d5962f2a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:41.022560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7fdfa-bb1e-489d-8502-6329d5962f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:41.022833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7fdfa-bb1e-489d-8502-6329d5962f2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:41.031573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:41.031922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7fdfa-bb1e-489d-8502-6329d5962f2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:41.033972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7fdfa-bb1e-489d-8502-6329d5962f2a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-fcc7fdfa-bb1e-489d-8502-6329d5962f2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:41.049667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:41.050689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 501/963] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:16:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:41.051701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7fdfa-bb1e-489d-8502-6329d5962f2a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:41.083776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f47af3-a6dd-4b42-b931-b655b9be0cee None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:41.121761 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba843e7f-a918-4e60-a87e-cb6bc6f5fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:41.122435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843e7f-a918-4e60-a87e-cb6bc6f5fa45 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:41.122723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843e7f-a918-4e60-a87e-cb6bc6f5fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:41.122991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba843e7f-a918-4e60-a87e-cb6bc6f5fa45 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:41.123737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba843e7f-a918-4e60-a87e-cb6bc6f5fa45 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:41.124046 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba843e7f-a918-4e60-a87e-cb6bc6f5fa45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:41.125421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba843e7f-a918-4e60-a87e-cb6bc6f5fa45 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:41.139578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 463/964] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:41.139578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba843e7f-a918-4e60-a87e-cb6bc6f5fa45 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:41.175476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7fdfa-bb1e-489d-8502-6329d5962f2a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:41.217427 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32308de3-b48f-4119-8105-b6e9be13e364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:41.218327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32308de3-b48f-4119-8105-b6e9be13e364 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:41.218725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32308de3-b48f-4119-8105-b6e9be13e364 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:41.219101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32308de3-b48f-4119-8105-b6e9be13e364 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:41.220138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32308de3-b48f-4119-8105-b6e9be13e364 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:41.220553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32308de3-b48f-4119-8105-b6e9be13e364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:41.222107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32308de3-b48f-4119-8105-b6e9be13e364 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:41.233193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 502/965] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:41.234299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32308de3-b48f-4119-8105-b6e9be13e364 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:41.261259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:41.270546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba843e7f-a918-4e60-a87e-cb6bc6f5fa45 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:41.308252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:41.325377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0220-ba26-479a-a045-6749f68603c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:41.326328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0220-ba26-479a-a045-6749f68603c5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:41.326815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0220-ba26-479a-a045-6749f68603c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:41.327112 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a0220-ba26-479a-a045-6749f68603c5 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:41.328711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a0220-ba26-479a-a045-6749f68603c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:41.330031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a0220-ba26-479a-a045-6749f68603c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:41.332229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a0220-ba26-479a-a045-6749f68603c5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:41.355090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 464/966] 10.222.0.64 () {56 vars in 1149 bytes} [Mon Jun 27 09:16:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:41.361318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0220-ba26-479a-a045-6749f68603c5 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:41.377756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32308de3-b48f-4119-8105-b6e9be13e364 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:41.425848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048ae6e-f741-4359-961e-c38b45de4ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:41.426721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048ae6e-f741-4359-961e-c38b45de4ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:41.426851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048ae6e-f741-4359-961e-c38b45de4ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:41.427114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048ae6e-f741-4359-961e-c38b45de4ce6 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:41.427858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048ae6e-f741-4359-961e-c38b45de4ce6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:41.428163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048ae6e-f741-4359-961e-c38b45de4ce6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:41.429631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048ae6e-f741-4359-961e-c38b45de4ce6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:41.439663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 503/967] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:41.441551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048ae6e-f741-4359-961e-c38b45de4ce6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:41.495784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a0220-ba26-479a-a045-6749f68603c5 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:41.543084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:41.543594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83d560-4a09-404c-902f-c6134e9f0095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:41.544567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83d560-4a09-404c-902f-c6134e9f0095 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:41.544851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83d560-4a09-404c-902f-c6134e9f0095 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:41.545130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83d560-4a09-404c-902f-c6134e9f0095 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:41.565402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048ae6e-f741-4359-961e-c38b45de4ce6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:41.610011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a1eaa-5934-4b6c-ab3c-ed4b372b90af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:41.611505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a1eaa-5934-4b6c-ab3c-ed4b372b90af None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:41.611650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:41.612099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a1eaa-5934-4b6c-ab3c-ed4b372b90af None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:41.612552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a1eaa-5934-4b6c-ab3c-ed4b372b90af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:41.613516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a1eaa-5934-4b6c-ab3c-ed4b372b90af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:41.615109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a1eaa-5934-4b6c-ab3c-ed4b372b90af None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:41.635619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83d560-4a09-404c-902f-c6134e9f0095 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:41.637173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83d560-4a09-404c-902f-c6134e9f0095 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:41.771691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 465/968] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:41 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:16:41.774579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83d560-4a09-404c-902f-c6134e9f0095 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:41.823859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 504/969] 10.222.0.64 () {60 vars in 1144 bytes} [Mon Jun 27 09:16:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:16:41.924403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:41.948867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83d560-4a09-404c-902f-c6134e9f0095 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:42.000657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26e309-7b9a-4b80-91a1-7d6b85d7ee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:42.001882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26e309-7b9a-4b80-91a1-7d6b85d7ee8b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:42.002365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26e309-7b9a-4b80-91a1-7d6b85d7ee8b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:42.002857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26e309-7b9a-4b80-91a1-7d6b85d7ee8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:42.015846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26e309-7b9a-4b80-91a1-7d6b85d7ee8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:42.017765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26e309-7b9a-4b80-91a1-7d6b85d7ee8b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-fe26e309-7b9a-4b80-91a1-7d6b85d7ee8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:42.039108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:42.040096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:42.041010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:42.041010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:42.041010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:42.041010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:42.044786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 466/970] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:42.053234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a1eaa-5934-4b6c-ab3c-ed4b372b90af None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:42.071056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2986f5-4b2e-4e1c-a7de-706c4bdeafff None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:42.071402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2986f5-4b2e-4e1c-a7de-706c4bdeafff None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:42.071648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2986f5-4b2e-4e1c-a7de-706c4bdeafff None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:42.073621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 467/971] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:42.078959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64baa0-bb3e-4d86-a1b2-0c28c44c2129 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:42.079266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64baa0-bb3e-4d86-a1b2-0c28c44c2129 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:42.079551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64baa0-bb3e-4d86-a1b2-0c28c44c2129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:42.098428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-2e64baa0-bb3e-4d86-a1b2-0c28c44c2129 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:42.099664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2e64baa0-bb3e-4d86-a1b2-0c28c44c2129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:42.176426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 468/972] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:42.201994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a1eaa-5934-4b6c-ab3c-ed4b372b90af None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:42.202626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64baa0-bb3e-4d86-a1b2-0c28c44c2129 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:42.256533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19e788-a62b-46a0-a36a-5167ecf3bbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:42.257862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19e788-a62b-46a0-a36a-5167ecf3bbff None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:42.258882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19e788-a62b-46a0-a36a-5167ecf3bbff None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:42.259502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19e788-a62b-46a0-a36a-5167ecf3bbff None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:42.260556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19e788-a62b-46a0-a36a-5167ecf3bbff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:42.261297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19e788-a62b-46a0-a36a-5167ecf3bbff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:42.263288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19e788-a62b-46a0-a36a-5167ecf3bbff None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:42.272158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:42.277868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:42.284067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 505/973] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:42.292762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19e788-a62b-46a0-a36a-5167ecf3bbff None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:42.331622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64baa0-bb3e-4d86-a1b2-0c28c44c2129 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:42.371501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:42.372240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:42.372566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:42.372851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:42.373595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:42.375101 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:42.419804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.sql.core [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 09:16:42.419804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 09:16:42.419804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [parameters: {'user_id': '30f25514ac4e4816b8fdd3a8c2152514', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 09:16:42.419804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80344) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application conn.query(q) Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result.read() Jun 27 09:16:42.429525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 09:16:42.430469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application next(self.gen) Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application yield resource Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application next(self.gen) Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise self.value Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application session.commit() Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 09:16:42.431422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.session.flush() Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application compat.raise_( Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 09:16:42.432729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 09:16:42.433795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application conn.query(q) Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application result.read() Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 09:16:42.434846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [parameters: {'user_id': '30f25514ac4e4816b8fdd3a8c2152514', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:42.435953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 27 09:16:42.437119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 27 09:16:42.438260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 27 09:16:42.438260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 27 09:16:42.438260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 09:16:42.438260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 09:16:42.438260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 09:16:42.438260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:42.438260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 469/974] 10.222.0.64 () {60 vars in 1131 bytes} [Mon Jun 27 09:16:42 2022] POST /identity/v3/users => generated 160 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:42.438750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19e788-a62b-46a0-a36a-5167ecf3bbff None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:42.449552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:42.492638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1eb148-d96a-440a-ac46-9f6229772ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:42.494379 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eb148-d96a-440a-ac46-9f6229772ebe None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:42.495089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eb148-d96a-440a-ac46-9f6229772ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:42.495853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eb148-d96a-440a-ac46-9f6229772ebe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:42.508966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eb148-d96a-440a-ac46-9f6229772ebe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:42.512075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eb148-d96a-440a-ac46-9f6229772ebe None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-ae1eb148-d96a-440a-ac46-9f6229772ebe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:42.540897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:42.542846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:42.544858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:42.544858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:42.544858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:42.544858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:42.544858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:42.544858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:42.544858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:42.544858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:42.548681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 506/975] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:42.556432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1eb148-d96a-440a-ac46-9f6229772ebe None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:42.600187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9555752-91a4-4b5c-9b79-0662cc0c8c6f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:42.623514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:42.636581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:42.642093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f41ab-6a46-41ee-a536-7fb8714c8016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:42.643325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f41ab-6a46-41ee-a536-7fb8714c8016 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:42.643851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f41ab-6a46-41ee-a536-7fb8714c8016 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:42.644400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f41ab-6a46-41ee-a536-7fb8714c8016 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:42.655476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f41ab-6a46-41ee-a536-7fb8714c8016 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:42.658424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f41ab-6a46-41ee-a536-7fb8714c8016 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-927f41ab-6a46-41ee-a536-7fb8714c8016 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:42.675058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:42.676135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 09:16:42.677071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:42.679943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 470/976] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:16:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:42.691279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f41ab-6a46-41ee-a536-7fb8714c8016 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:42.729480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1eb148-d96a-440a-ac46-9f6229772ebe None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:42.797927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92766e-8863-4365-97af-64749e63fb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:42.799072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92766e-8863-4365-97af-64749e63fb41 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:42.799620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92766e-8863-4365-97af-64749e63fb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:42.800110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92766e-8863-4365-97af-64749e63fb41 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:42.801626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92766e-8863-4365-97af-64749e63fb41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:42.802186 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92766e-8863-4365-97af-64749e63fb41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:42.805041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92766e-8863-4365-97af-64749e63fb41 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:42.817918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f41ab-6a46-41ee-a536-7fb8714c8016 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:42.835619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 507/977] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:42.838367 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92766e-8863-4365-97af-64749e63fb41 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:42.874847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5cd24-9995-460d-9109-be1d2f874091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:42.876705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5cd24-9995-460d-9109-be1d2f874091 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:42.877220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5cd24-9995-460d-9109-be1d2f874091 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:42.877778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5cd24-9995-460d-9109-be1d2f874091 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:42.878811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5cd24-9995-460d-9109-be1d2f874091 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:42.879324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5cd24-9995-460d-9109-be1d2f874091 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:42.881310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5cd24-9995-460d-9109-be1d2f874091 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:42.926941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 471/978] 10.222.0.64 () {56 vars in 1101 bytes} [Mon Jun 27 09:16:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:43.049857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92766e-8863-4365-97af-64749e63fb41 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:43.091671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:43.120005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d67db5-271c-481d-aad3-a49173972030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:43.121228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d67db5-271c-481d-aad3-a49173972030 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:43.121743 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d67db5-271c-481d-aad3-a49173972030 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:43.122248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d67db5-271c-481d-aad3-a49173972030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:43.123730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d67db5-271c-481d-aad3-a49173972030 None admin] RBAC: Adding query filter params (role.id=e0d6e1d5614e4a4881a0320bda7328cb, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=8731ac7e5a7941f1b2ad04bddc966db6) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:43.124297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d67db5-271c-481d-aad3-a49173972030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:43.127814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d67db5-271c-481d-aad3-a49173972030 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:43.140420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 508/979] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:42 2022] GET /identity/v3/role_assignments?user.id=8731ac7e5a7941f1b2ad04bddc966db6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=e0d6e1d5614e4a4881a0320bda7328cb => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:43.347637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ed34b-a716-49df-8557-da0386728471 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:43.347836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ed34b-a716-49df-8557-da0386728471 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:43.348116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ed34b-a716-49df-8557-da0386728471 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:43.349722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 472/980] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:43.355552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196304d-3c2e-4b6b-bc17-a0be8dda1fc7 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:43.355848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196304d-3c2e-4b6b-bc17-a0be8dda1fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:43.356297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196304d-3c2e-4b6b-bc17-a0be8dda1fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:43.374906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e196304d-3c2e-4b6b-bc17-a0be8dda1fc7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:43.376776 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e196304d-3c2e-4b6b-bc17-a0be8dda1fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:43.465604 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 509/981] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:43.489704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ed34b-a716-49df-8557-da0386728471 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:43.531498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:43.614357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ed34b-a716-49df-8557-da0386728471 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:43.649698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba76e3-e6f2-4652-8e92-a043f4d14640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:43.651025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba76e3-e6f2-4652-8e92-a043f4d14640 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:43.651549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba76e3-e6f2-4652-8e92-a043f4d14640 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:43.652034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba76e3-e6f2-4652-8e92-a043f4d14640 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:43.653035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba76e3-e6f2-4652-8e92-a043f4d14640 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:43.654682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba76e3-e6f2-4652-8e92-a043f4d14640 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:43.733217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 473/982] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:16:43 2022] POST /identity/v3/services => generated 242 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:16:44.134884 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270d5eb-ec4d-4e5a-af4f-420f68236c2f None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:44.135894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270d5eb-ec4d-4e5a-af4f-420f68236c2f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:44.135894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270d5eb-ec4d-4e5a-af4f-420f68236c2f None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:44.137023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 510/983] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:44.143498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096cdfa-a0e7-4a59-96a2-6f4c7062d845 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:44.144228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096cdfa-a0e7-4a59-96a2-6f4c7062d845 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:44.144228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096cdfa-a0e7-4a59-96a2-6f4c7062d845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:44.164760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7096cdfa-a0e7-4a59-96a2-6f4c7062d845 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:44.166329 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7096cdfa-a0e7-4a59-96a2-6f4c7062d845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:44.258668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 474/984] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:44.301215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270d5eb-ec4d-4e5a-af4f-420f68236c2f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:44.489301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270d5eb-ec4d-4e5a-af4f-420f68236c2f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:44.510916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:44.554061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c84992-6787-4c74-b506-710f3824a4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:44.554999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c84992-6787-4c74-b506-710f3824a4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:44.555281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c84992-6787-4c74-b506-710f3824a4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:44.555571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c84992-6787-4c74-b506-710f3824a4a5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:44.556245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c84992-6787-4c74-b506-710f3824a4a5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:44.557625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c84992-6787-4c74-b506-710f3824a4a5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-36c84992-6787-4c74-b506-710f3824a4a5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:44.571339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:44.572618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:44.578563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 511/985] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:44.581017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096cdfa-a0e7-4a59-96a2-6f4c7062d845 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:44.616476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:44.703870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096cdfa-a0e7-4a59-96a2-6f4c7062d845 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:44.743185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84587f6-78a7-4e39-a631-074a16b35278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:44.743861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84587f6-78a7-4e39-a631-074a16b35278 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:44.744141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84587f6-78a7-4e39-a631-074a16b35278 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:44.744413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84587f6-78a7-4e39-a631-074a16b35278 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:44.745149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84587f6-78a7-4e39-a631-074a16b35278 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:44.745453 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84587f6-78a7-4e39-a631-074a16b35278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:44.746847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84587f6-78a7-4e39-a631-074a16b35278 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:44.753546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 475/986] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:44.759738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c84992-6787-4c74-b506-710f3824a4a5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:44.884005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c84992-6787-4c74-b506-710f3824a4a5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:44.925375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf351784-a233-4fa8-b6c1-9f6df1aa80c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:44.926156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf351784-a233-4fa8-b6c1-9f6df1aa80c0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:44.926447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf351784-a233-4fa8-b6c1-9f6df1aa80c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:44.926712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf351784-a233-4fa8-b6c1-9f6df1aa80c0 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:44.927447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf351784-a233-4fa8-b6c1-9f6df1aa80c0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:44.927785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf351784-a233-4fa8-b6c1-9f6df1aa80c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:44.929200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf351784-a233-4fa8-b6c1-9f6df1aa80c0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:44.942538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 512/987] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:44.943841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84587f6-78a7-4e39-a631-074a16b35278 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:45.046052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:45.062670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84587f6-78a7-4e39-a631-074a16b35278 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:45.118223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccbfd4-8516-46ac-b1cc-e42104fcb613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:45.118896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccbfd4-8516-46ac-b1cc-e42104fcb613 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:45.119181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccbfd4-8516-46ac-b1cc-e42104fcb613 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:45.119459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccbfd4-8516-46ac-b1cc-e42104fcb613 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:45.120320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccbfd4-8516-46ac-b1cc-e42104fcb613 None admin] RBAC: Adding query filter params (interface=admin, service_id=a3856a8940324d1f979dd592472b1b5b, region_id=RegionOne) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:45.120635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccbfd4-8516-46ac-b1cc-e42104fcb613 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:45.122029 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccbfd4-8516-46ac-b1cc-e42104fcb613 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:45.132698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 476/988] 10.222.0.64 () {56 vars in 1249 bytes} [Mon Jun 27 09:16:44 2022] GET /identity/v3/endpoints?service_id=a3856a8940324d1f979dd592472b1b5b&interface=admin®ion_id=RegionOne => generated 193 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:45.133939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf351784-a233-4fa8-b6c1-9f6df1aa80c0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:45.181491 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:45.269071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf351784-a233-4fa8-b6c1-9f6df1aa80c0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:45.336242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf39a1e-b499-47cf-83eb-9a1f84f9a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:45.337318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf39a1e-b499-47cf-83eb-9a1f84f9a0bf None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:45.337834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf39a1e-b499-47cf-83eb-9a1f84f9a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:45.338475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf39a1e-b499-47cf-83eb-9a1f84f9a0bf None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:45.339849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf39a1e-b499-47cf-83eb-9a1f84f9a0bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:45.340619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d517dd-6646-4793-9fab-c0ce0b652635 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:45.341347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d517dd-6646-4793-9fab-c0ce0b652635 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:45.341347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d517dd-6646-4793-9fab-c0ce0b652635 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:45.342489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf39a1e-b499-47cf-83eb-9a1f84f9a0bf None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:45.342832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 477/989] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:45.348213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f6e8-4341-408c-af97-830b8924b76f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:45.348526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f6e8-4341-408c-af97-830b8924b76f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:45.348793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f6e8-4341-408c-af97-830b8924b76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:45.356673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 513/990] 10.222.0.64 () {56 vars in 1087 bytes} [Mon Jun 27 09:16:45 2022] GET /identity/v3/services => generated 1727 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:16:45.373613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-49b4f6e8-4341-408c-af97-830b8924b76f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:45.376688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-49b4f6e8-4341-408c-af97-830b8924b76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:45.484212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 478/991] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:45.507926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf39a1e-b499-47cf-83eb-9a1f84f9a0bf None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:45.692060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf39a1e-b499-47cf-83eb-9a1f84f9a0bf None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:45.708762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:45.757476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c21d9a-e9a5-4f38-b8f9-474eba47a8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:45.758282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c21d9a-e9a5-4f38-b8f9-474eba47a8db None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:45.758613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c21d9a-e9a5-4f38-b8f9-474eba47a8db None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:45.758891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c21d9a-e9a5-4f38-b8f9-474eba47a8db None admin] PATH_INFO: `/v3/services/image` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:45.759584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c21d9a-e9a5-4f38-b8f9-474eba47a8db None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:45.761077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c21d9a-e9a5-4f38-b8f9-474eba47a8db None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-e1c21d9a-e9a5-4f38-b8f9-474eba47a8db None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:45.773659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:45.774880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:45.777083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 514/992] 10.222.0.64 () {56 vars in 1105 bytes} [Mon Jun 27 09:16:45 2022] GET /identity/v3/services/image => generated 86 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:45.788509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f6e8-4341-408c-af97-830b8924b76f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:45.880879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:45.900076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f6e8-4341-408c-af97-830b8924b76f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:45.939196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4406aeb-5028-4994-971d-4cd221ba7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:45.939939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406aeb-5028-4994-971d-4cd221ba7455 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:45.940279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406aeb-5028-4994-971d-4cd221ba7455 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:45.940690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406aeb-5028-4994-971d-4cd221ba7455 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:45.941454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4406aeb-5028-4994-971d-4cd221ba7455 None admin] RBAC: Adding query filter params (name=image) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:45.941850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4406aeb-5028-4994-971d-4cd221ba7455 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:45.943372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4406aeb-5028-4994-971d-4cd221ba7455 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:45.949430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 479/993] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:45.955581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c21d9a-e9a5-4f38-b8f9-474eba47a8db None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:46.061801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c067f-41e4-4c1b-a227-168d6b1280ee None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:46.062187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c067f-41e4-4c1b-a227-168d6b1280ee None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:46.062438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c067f-41e4-4c1b-a227-168d6b1280ee None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:46.064135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 480/994] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:46.069707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db25bb1-5191-485d-bc9f-ed29c1bbff2f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:46.069991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db25bb1-5191-485d-bc9f-ed29c1bbff2f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:46.070260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db25bb1-5191-485d-bc9f-ed29c1bbff2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:46.088670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4db25bb1-5191-485d-bc9f-ed29c1bbff2f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:46.090413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4db25bb1-5191-485d-bc9f-ed29c1bbff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:46.112299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c21d9a-e9a5-4f38-b8f9-474eba47a8db None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:46.156888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48e79-32c1-48d5-ae5d-e98bde979684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:46.158357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48e79-32c1-48d5-ae5d-e98bde979684 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:46.158830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48e79-32c1-48d5-ae5d-e98bde979684 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:46.159224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48e79-32c1-48d5-ae5d-e98bde979684 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:46.160335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d48e79-32c1-48d5-ae5d-e98bde979684 None admin] RBAC: Adding query filter params (type=image) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:46.160781 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d48e79-32c1-48d5-ae5d-e98bde979684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:46.162320 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d48e79-32c1-48d5-ae5d-e98bde979684 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:46.169154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 515/995] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:46.203731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 481/996] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:46.242696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48e79-32c1-48d5-ae5d-e98bde979684 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:46.294421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:46.388384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48e79-32c1-48d5-ae5d-e98bde979684 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:46.431233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ae9b4-a785-4b66-9709-1dbbb7d18a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:46.432092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ae9b4-a785-4b66-9709-1dbbb7d18a30 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:46.432371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ae9b4-a785-4b66-9709-1dbbb7d18a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:46.432658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ae9b4-a785-4b66-9709-1dbbb7d18a30 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:46.433308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ae9b4-a785-4b66-9709-1dbbb7d18a30 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:46.434720 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ae9b4-a785-4b66-9709-1dbbb7d18a30 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-519ae9b4-a785-4b66-9709-1dbbb7d18a30 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:46.448030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:46.456722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:46.458704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 516/997] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:46.464797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db25bb1-5191-485d-bc9f-ed29c1bbff2f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:46.596772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:46.632059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db25bb1-5191-485d-bc9f-ed29c1bbff2f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:46.670766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6831934-9832-45ab-9798-194a46f9b6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:46.671447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6831934-9832-45ab-9798-194a46f9b6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:46.671753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6831934-9832-45ab-9798-194a46f9b6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:46.672027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6831934-9832-45ab-9798-194a46f9b6c4 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:46.672823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6831934-9832-45ab-9798-194a46f9b6c4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:46.673621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6831934-9832-45ab-9798-194a46f9b6c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:46.675042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6831934-9832-45ab-9798-194a46f9b6c4 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:46.682852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 482/998] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:46.688715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ae9b4-a785-4b66-9709-1dbbb7d18a30 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:46.862946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:46.871007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ae9b4-a785-4b66-9709-1dbbb7d18a30 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:46.939465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc3811-427d-4359-898d-b47a02d74196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:46.940484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc3811-427d-4359-898d-b47a02d74196 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:46.940993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc3811-427d-4359-898d-b47a02d74196 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:46.941488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc3811-427d-4359-898d-b47a02d74196 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:46.942848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc3811-427d-4359-898d-b47a02d74196 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:46.943398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc3811-427d-4359-898d-b47a02d74196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:46.946142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc3811-427d-4359-898d-b47a02d74196 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:46.956210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 517/999] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:46.965821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6831934-9832-45ab-9798-194a46f9b6c4 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:47.152964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6831934-9832-45ab-9798-194a46f9b6c4 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:47.199110 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0839e-915f-4444-bffc-2f8beedb4a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:47.200674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0839e-915f-4444-bffc-2f8beedb4a12 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:47.205767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0839e-915f-4444-bffc-2f8beedb4a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:47.205767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0839e-915f-4444-bffc-2f8beedb4a12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:47.205767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0839e-915f-4444-bffc-2f8beedb4a12 None admin] RBAC: Adding query filter params (interface=public, service_id=7838c061cc76485da3372289556af4da, region_id=RegionOne) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:47.205767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0839e-915f-4444-bffc-2f8beedb4a12 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:47.207496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0839e-915f-4444-bffc-2f8beedb4a12 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:47.218867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 483/1000] 10.222.0.64 () {56 vars in 1251 bytes} [Mon Jun 27 09:16:46 2022] GET /identity/v3/endpoints?service_id=7838c061cc76485da3372289556af4da&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:47.224254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc3811-427d-4359-898d-b47a02d74196 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:47.370664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc3811-427d-4359-898d-b47a02d74196 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:47.410242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e1dd7-befe-432c-8ba0-57d63570ca19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:47.410829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e1dd7-befe-432c-8ba0-57d63570ca19 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:47.411112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e1dd7-befe-432c-8ba0-57d63570ca19 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:47.411389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e1dd7-befe-432c-8ba0-57d63570ca19 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:47.412105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e1dd7-befe-432c-8ba0-57d63570ca19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:47.413511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e1dd7-befe-432c-8ba0-57d63570ca19 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:47.431244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 518/1001] 10.222.0.64 () {56 vars in 1087 bytes} [Mon Jun 27 09:16:47 2022] GET /identity/v3/services => generated 1727 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:16:47.564445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02b8f-9926-4d8d-a11d-0e238420342e None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:47.565401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02b8f-9926-4d8d-a11d-0e238420342e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:47.565401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02b8f-9926-4d8d-a11d-0e238420342e None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:47.566755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 484/1002] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:47.573587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27baef-0768-4b14-99cc-b96920180fdd None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:47.574094 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27baef-0768-4b14-99cc-b96920180fdd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:47.574597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27baef-0768-4b14-99cc-b96920180fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:47.606955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ae27baef-0768-4b14-99cc-b96920180fdd None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:47.610971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ae27baef-0768-4b14-99cc-b96920180fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:47.645901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:47.713533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 519/1003] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:47.737709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b02b8f-9926-4d8d-a11d-0e238420342e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:47.742662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:47.869875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b02b8f-9926-4d8d-a11d-0e238420342e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:47.912108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6b60-0c01-4398-b9e7-5d5f22a5ad92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:47.913192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6b60-0c01-4398-b9e7-5d5f22a5ad92 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:47.913629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6b60-0c01-4398-b9e7-5d5f22a5ad92 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:47.914091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6b60-0c01-4398-b9e7-5d5f22a5ad92 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:47.915004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f6b60-0c01-4398-b9e7-5d5f22a5ad92 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:47.916682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f6b60-0c01-4398-b9e7-5d5f22a5ad92 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-ee1f6b60-0c01-4398-b9e7-5d5f22a5ad92 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:47.930561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:47.931822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 09:16:47.932721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:47.933852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 485/1004] 10.222.0.64 () {56 vars in 1126 bytes} [Mon Jun 27 09:16:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:47.944785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27baef-0768-4b14-99cc-b96920180fdd None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:48.096784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27baef-0768-4b14-99cc-b96920180fdd None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:48.137955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d50452-be7c-45d2-ac46-4370a423a054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:48.138636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d50452-be7c-45d2-ac46-4370a423a054 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:48.138913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d50452-be7c-45d2-ac46-4370a423a054 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:48.139189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d50452-be7c-45d2-ac46-4370a423a054 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:48.140030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d50452-be7c-45d2-ac46-4370a423a054 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:48.140341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d50452-be7c-45d2-ac46-4370a423a054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:48.142314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d50452-be7c-45d2-ac46-4370a423a054 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:48.148766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 520/1005] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:48.154878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6b60-0c01-4398-b9e7-5d5f22a5ad92 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:48.233156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:48.288205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6b60-0c01-4398-b9e7-5d5f22a5ad92 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:48.326015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ede1d-c0a8-49bf-8cdc-771fba9521f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:48.326796 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ede1d-c0a8-49bf-8cdc-771fba9521f8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:48.327043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ede1d-c0a8-49bf-8cdc-771fba9521f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:48.327287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ede1d-c0a8-49bf-8cdc-771fba9521f8 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:48.328076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ede1d-c0a8-49bf-8cdc-771fba9521f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:48.328389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ede1d-c0a8-49bf-8cdc-771fba9521f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:48.329858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ede1d-c0a8-49bf-8cdc-771fba9521f8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:48.343013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 486/1006] 10.222.0.64 () {56 vars in 1122 bytes} [Mon Jun 27 09:16:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:48.345696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d50452-be7c-45d2-ac46-4370a423a054 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:48.386260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:48.405162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b5e62-f012-4131-9b1a-7372aadd3cdb None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:48.405538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b5e62-f012-4131-9b1a-7372aadd3cdb None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:48.406404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b5e62-f012-4131-9b1a-7372aadd3cdb None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:48.407172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 487/1007] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:48.414052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b30bb3-f077-450c-b8b1-b8759160a4d2 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:48.414052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b30bb3-f077-450c-b8b1-b8759160a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:48.414052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b30bb3-f077-450c-b8b1-b8759160a4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:48.433361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f1b30bb3-f077-450c-b8b1-b8759160a4d2 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:48.434563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f1b30bb3-f077-450c-b8b1-b8759160a4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:48.487815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d50452-be7c-45d2-ac46-4370a423a054 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:48.516495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 488/1008] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:48.524752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af48b1-b06a-45e1-92e6-fa17a6e9f15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:48.525429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af48b1-b06a-45e1-92e6-fa17a6e9f15b None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:48.525965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af48b1-b06a-45e1-92e6-fa17a6e9f15b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:48.526244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af48b1-b06a-45e1-92e6-fa17a6e9f15b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:48.527082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af48b1-b06a-45e1-92e6-fa17a6e9f15b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:48.528518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af48b1-b06a-45e1-92e6-fa17a6e9f15b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:48.539857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b30bb3-f077-450c-b8b1-b8759160a4d2 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:48.604007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 521/1009] 10.222.0.64 () {60 vars in 1144 bytes} [Mon Jun 27 09:16:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:16:48.616799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:48.684091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b30bb3-f077-450c-b8b1-b8759160a4d2 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:48.757681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06465ca0-2fe1-45a8-9f67-0324df3eade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:48.758915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465ca0-2fe1-45a8-9f67-0324df3eade6 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:48.759326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465ca0-2fe1-45a8-9f67-0324df3eade6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:48.759849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465ca0-2fe1-45a8-9f67-0324df3eade6 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:48.761169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06465ca0-2fe1-45a8-9f67-0324df3eade6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:48.763830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06465ca0-2fe1-45a8-9f67-0324df3eade6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:48.880148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 489/1010] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:16:48 2022] POST /identity/v3/services => generated 242 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:16:49.572009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918811f-9336-45cb-be1a-e57d6640b88a None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:49.572594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918811f-9336-45cb-be1a-e57d6640b88a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:49.572594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918811f-9336-45cb-be1a-e57d6640b88a None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:49.574624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 522/1011] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:49.580545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029462e5-2a36-417b-88ef-3acfd88a28fd None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:49.580926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029462e5-2a36-417b-88ef-3acfd88a28fd None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:49.581083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029462e5-2a36-417b-88ef-3acfd88a28fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:49.607027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-029462e5-2a36-417b-88ef-3acfd88a28fd None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:49.609057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-029462e5-2a36-417b-88ef-3acfd88a28fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:49.706183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 490/1012] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:49 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:49.731013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918811f-9336-45cb-be1a-e57d6640b88a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:49.864575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:49.893747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918811f-9336-45cb-be1a-e57d6640b88a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:49.942878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e9ac-588d-4222-a116-02821375391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:49.943728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e9ac-588d-4222-a116-02821375391f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:49.944011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e9ac-588d-4222-a116-02821375391f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:49.944288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e9ac-588d-4222-a116-02821375391f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:49.944960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16e9ac-588d-4222-a116-02821375391f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:49.946359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16e9ac-588d-4222-a116-02821375391f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-2b16e9ac-588d-4222-a116-02821375391f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:49.959914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:49.961081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:49.962140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 523/1013] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:49.967395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029462e5-2a36-417b-88ef-3acfd88a28fd None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:50.019632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:50.084170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029462e5-2a36-417b-88ef-3acfd88a28fd None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:50.120007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698ea5-85ac-49fa-a459-b9cd763e26fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:50.120626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698ea5-85ac-49fa-a459-b9cd763e26fa None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:50.120907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698ea5-85ac-49fa-a459-b9cd763e26fa None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:50.121179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698ea5-85ac-49fa-a459-b9cd763e26fa None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:50.122126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58698ea5-85ac-49fa-a459-b9cd763e26fa None admin] RBAC: Adding query filter params (name=compute) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:50.122608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58698ea5-85ac-49fa-a459-b9cd763e26fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:50.124663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58698ea5-85ac-49fa-a459-b9cd763e26fa None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:50.134742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 491/1014] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:50.136526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e9ac-588d-4222-a116-02821375391f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:50.259256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e9ac-588d-4222-a116-02821375391f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:50.318475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6524216-c2be-4ee6-858a-17a28c85ec2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:50.319669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6524216-c2be-4ee6-858a-17a28c85ec2c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:50.320194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6524216-c2be-4ee6-858a-17a28c85ec2c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:50.320709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6524216-c2be-4ee6-858a-17a28c85ec2c None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:50.322063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6524216-c2be-4ee6-858a-17a28c85ec2c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:50.322595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6524216-c2be-4ee6-858a-17a28c85ec2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:50.325125 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6524216-c2be-4ee6-858a-17a28c85ec2c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:50.339374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 524/1015] 10.222.0.64 () {56 vars in 1112 bytes} [Mon Jun 27 09:16:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:50.340798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698ea5-85ac-49fa-a459-b9cd763e26fa None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:50.464573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698ea5-85ac-49fa-a459-b9cd763e26fa None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:50.476362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:50.505383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adafba-7b4e-44d1-818b-7433f5e2e659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:50.506222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adafba-7b4e-44d1-818b-7433f5e2e659 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:50.506509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adafba-7b4e-44d1-818b-7433f5e2e659 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:50.506793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adafba-7b4e-44d1-818b-7433f5e2e659 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:50.507585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adafba-7b4e-44d1-818b-7433f5e2e659 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:50.509010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adafba-7b4e-44d1-818b-7433f5e2e659 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:50.606034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 492/1016] 10.222.0.64 () {60 vars in 1144 bytes} [Mon Jun 27 09:16:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:16:50.750438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093c9f8-06d1-4f93-9c8f-a7418c91f300 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:50.751634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093c9f8-06d1-4f93-9c8f-a7418c91f300 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:50.751634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093c9f8-06d1-4f93-9c8f-a7418c91f300 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:50.753361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 525/1017] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:50.759417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fa51f-673e-4edc-b29e-5fd9e8d82c18 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:50.760289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fa51f-673e-4edc-b29e-5fd9e8d82c18 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:50.760289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fa51f-673e-4edc-b29e-5fd9e8d82c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:50.782981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-899fa51f-673e-4edc-b29e-5fd9e8d82c18 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:50.785166 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-899fa51f-673e-4edc-b29e-5fd9e8d82c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:50.892582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 493/1018] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:50.918525 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093c9f8-06d1-4f93-9c8f-a7418c91f300 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:50.931689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:51.012516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c346a-24ae-4169-a81d-110e5f0aa271 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:51.012943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c346a-24ae-4169-a81d-110e5f0aa271 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:51.013411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c346a-24ae-4169-a81d-110e5f0aa271 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:51.016177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 494/1019] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:51.022026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b301a-b279-47c4-b36b-27db219a0028 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:51.022410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b301a-b279-47c4-b36b-27db219a0028 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:51.022595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b301a-b279-47c4-b36b-27db219a0028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:51.043096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-551b301a-b279-47c4-b36b-27db219a0028 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:51.044373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-551b301a-b279-47c4-b36b-27db219a0028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:51.045436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093c9f8-06d1-4f93-9c8f-a7418c91f300 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:51.087464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706193a4-3dc2-468a-874e-7ab11c22d182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:51.088537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706193a4-3dc2-468a-874e-7ab11c22d182 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:51.088973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706193a4-3dc2-468a-874e-7ab11c22d182 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:51.089339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706193a4-3dc2-468a-874e-7ab11c22d182 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:51.090667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706193a4-3dc2-468a-874e-7ab11c22d182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:51.092168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706193a4-3dc2-468a-874e-7ab11c22d182 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:51.093879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 526/1020] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:51.102831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706193a4-3dc2-468a-874e-7ab11c22d182 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:51.146685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 495/1021] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:51.170182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b301a-b279-47c4-b36b-27db219a0028 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:51.200891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:51.243875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:51.279248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706193a4-3dc2-468a-874e-7ab11c22d182 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:51.323322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b301a-b279-47c4-b36b-27db219a0028 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:51.333615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e59659-08d4-4820-be4b-211b41137c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:51.335159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e59659-08d4-4820-be4b-211b41137c72 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:51.335975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e59659-08d4-4820-be4b-211b41137c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:51.336710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e59659-08d4-4820-be4b-211b41137c72 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:51.337974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e59659-08d4-4820-be4b-211b41137c72 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:51.340818 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e59659-08d4-4820-be4b-211b41137c72 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:51.376102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18403d-f1cd-488e-b049-7d6e354e6a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:51.383407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18403d-f1cd-488e-b049-7d6e354e6a1e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:51.383407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18403d-f1cd-488e-b049-7d6e354e6a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:51.383407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18403d-f1cd-488e-b049-7d6e354e6a1e None admin] PATH_INFO: `/v3/services/image` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:51.383407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18403d-f1cd-488e-b049-7d6e354e6a1e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:51.383407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18403d-f1cd-488e-b049-7d6e354e6a1e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-bd18403d-f1cd-488e-b049-7d6e354e6a1e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:51.392709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:51.393755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:51.394598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 496/1022] 10.222.0.64 () {56 vars in 1105 bytes} [Mon Jun 27 09:16:51 2022] GET /identity/v3/services/image => generated 86 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:51.400321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18403d-f1cd-488e-b049-7d6e354e6a1e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:51.452101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 527/1023] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:16:51 2022] POST /identity/v3/projects => generated 311 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:16:51.510665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:51.545373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18403d-f1cd-488e-b049-7d6e354e6a1e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:51.587389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd3b3a-4d3d-4c16-bcef-25730ddda44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:51.588202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd3b3a-4d3d-4c16-bcef-25730ddda44c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:51.588489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd3b3a-4d3d-4c16-bcef-25730ddda44c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:51.588770 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd3b3a-4d3d-4c16-bcef-25730ddda44c None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:51.589518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd3b3a-4d3d-4c16-bcef-25730ddda44c None admin] RBAC: Adding query filter params (name=image) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:51.589826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd3b3a-4d3d-4c16-bcef-25730ddda44c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:51.591248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd3b3a-4d3d-4c16-bcef-25730ddda44c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:51.602941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 497/1024] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:51.605029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e59659-08d4-4820-be4b-211b41137c72 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:51.724389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e59659-08d4-4820-be4b-211b41137c72 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:51.756011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:51.764184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d66381-fcbb-4f00-8472-1eacbc1421d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:51.764857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d66381-fcbb-4f00-8472-1eacbc1421d4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:51.769377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d66381-fcbb-4f00-8472-1eacbc1421d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:51.769377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d66381-fcbb-4f00-8472-1eacbc1421d4 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:51.769377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d66381-fcbb-4f00-8472-1eacbc1421d4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:51.769377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d66381-fcbb-4f00-8472-1eacbc1421d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:51.769377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d66381-fcbb-4f00-8472-1eacbc1421d4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:51.774366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 528/1025] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:51.781095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd3b3a-4d3d-4c16-bcef-25730ddda44c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:51.922274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd3b3a-4d3d-4c16-bcef-25730ddda44c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:51.962265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202e3e-2200-4931-a4ee-04b0ddd7daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:51.962939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202e3e-2200-4931-a4ee-04b0ddd7daa6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:51.963215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202e3e-2200-4931-a4ee-04b0ddd7daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:51.963485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202e3e-2200-4931-a4ee-04b0ddd7daa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:51.964331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202e3e-2200-4931-a4ee-04b0ddd7daa6 None admin] RBAC: Adding query filter params (interface=public, service_id=a1030154617f4063b8d0771111285358, region_id=RegionOne) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:51.964651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202e3e-2200-4931-a4ee-04b0ddd7daa6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:51.966128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202e3e-2200-4931-a4ee-04b0ddd7daa6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:51.972549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 498/1026] 10.222.0.64 () {56 vars in 1251 bytes} [Mon Jun 27 09:16:51 2022] GET /identity/v3/endpoints?service_id=a1030154617f4063b8d0771111285358&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:51.977903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d66381-fcbb-4f00-8472-1eacbc1421d4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:52.095580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d66381-fcbb-4f00-8472-1eacbc1421d4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:52.135068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5563da5-7b6d-4bd5-99c8-bebd1aa395fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:52.135764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5563da5-7b6d-4bd5-99c8-bebd1aa395fa None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:52.136043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5563da5-7b6d-4bd5-99c8-bebd1aa395fa None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:52.136317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5563da5-7b6d-4bd5-99c8-bebd1aa395fa None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:52.136997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5563da5-7b6d-4bd5-99c8-bebd1aa395fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:52.138385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5563da5-7b6d-4bd5-99c8-bebd1aa395fa None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:52.146558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 529/1027] 10.222.0.64 () {56 vars in 1087 bytes} [Mon Jun 27 09:16:51 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:16:52.955227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9617faa-6d6d-4c74-8223-f9fde8f5a67f None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:52.956157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9617faa-6d6d-4c74-8223-f9fde8f5a67f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:52.956157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9617faa-6d6d-4c74-8223-f9fde8f5a67f None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:52.958137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 499/1028] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:52.965131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b86e4d-e67f-4b61-971b-18267e2b40aa None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:52.965702 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b86e4d-e67f-4b61-971b-18267e2b40aa None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:52.965948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b86e4d-e67f-4b61-971b-18267e2b40aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:52.985155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d4b86e4d-e67f-4b61-971b-18267e2b40aa None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:52.986366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d4b86e4d-e67f-4b61-971b-18267e2b40aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:53.047166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:53.065479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 530/1029] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:53.090179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9617faa-6d6d-4c74-8223-f9fde8f5a67f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:53.120096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:53.298825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9617faa-6d6d-4c74-8223-f9fde8f5a67f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:53.338467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646f430-bb73-4c27-bf06-2110b8c201ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:53.339294 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646f430-bb73-4c27-bf06-2110b8c201ac None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:53.339602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646f430-bb73-4c27-bf06-2110b8c201ac None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:53.339964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646f430-bb73-4c27-bf06-2110b8c201ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:53.355388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646f430-bb73-4c27-bf06-2110b8c201ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-2646f430-bb73-4c27-bf06-2110b8c201ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:53.361800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:53.364512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:53.366449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 500/1030] 10.222.0.64 () {56 vars in 1120 bytes} [Mon Jun 27 09:16:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:53.369356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b86e4d-e67f-4b61-971b-18267e2b40aa None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:53.505112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b86e4d-e67f-4b61-971b-18267e2b40aa None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:53.569724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41be914-524d-4165-8b39-d59ae335e090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:53.570815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41be914-524d-4165-8b39-d59ae335e090 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:53.571327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41be914-524d-4165-8b39-d59ae335e090 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:53.571864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41be914-524d-4165-8b39-d59ae335e090 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:53.573350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41be914-524d-4165-8b39-d59ae335e090 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:53.573919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41be914-524d-4165-8b39-d59ae335e090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:53.576607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41be914-524d-4165-8b39-d59ae335e090 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:53.593068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 531/1031] 10.222.0.64 () {56 vars in 1115 bytes} [Mon Jun 27 09:16:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:53.602715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646f430-bb73-4c27-bf06-2110b8c201ac None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:53.634302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d1e95-c793-4fef-a82a-5692f278fb1a None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:53.634661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d1e95-c793-4fef-a82a-5692f278fb1a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:53.634873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d1e95-c793-4fef-a82a-5692f278fb1a None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:53.636301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 532/1032] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:53.641702 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80d083-e90c-4ce0-b7e7-ee2b229a72ae None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:53.641993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80d083-e90c-4ce0-b7e7-ee2b229a72ae None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:53.642272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80d083-e90c-4ce0-b7e7-ee2b229a72ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:53.659872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-9c80d083-e90c-4ce0-b7e7-ee2b229a72ae None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:53.661079 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9c80d083-e90c-4ce0-b7e7-ee2b229a72ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:53.712805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:53.727070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:53.755581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 533/1033] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:53.765260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646f430-bb73-4c27-bf06-2110b8c201ac None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:53.777534 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80d083-e90c-4ce0-b7e7-ee2b229a72ae None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:53.803775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c6f99-cbe7-4015-ad6d-5380e8c3d368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:53.804552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c6f99-cbe7-4015-ad6d-5380e8c3d368 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:53.804845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c6f99-cbe7-4015-ad6d-5380e8c3d368 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:53.805118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c6f99-cbe7-4015-ad6d-5380e8c3d368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:53.818212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c6f99-cbe7-4015-ad6d-5380e8c3d368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:53.819719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c6f99-cbe7-4015-ad6d-5380e8c3d368 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0f1c6f99-cbe7-4015-ad6d-5380e8c3d368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:53.839094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:53.840172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:53.841061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:53.841061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:53.841061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:53.841061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:53.841061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 501/1034] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:53.847028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c6f99-cbe7-4015-ad6d-5380e8c3d368 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:53.905837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80d083-e90c-4ce0-b7e7-ee2b229a72ae None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:53.949203 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218339a-03cc-4295-9d6e-2faec1676404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:53.950025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218339a-03cc-4295-9d6e-2faec1676404 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:53.950310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218339a-03cc-4295-9d6e-2faec1676404 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:53.950590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218339a-03cc-4295-9d6e-2faec1676404 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:53.952017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218339a-03cc-4295-9d6e-2faec1676404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:53.953492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218339a-03cc-4295-9d6e-2faec1676404 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:53.955230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 534/1035] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:53.961023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218339a-03cc-4295-9d6e-2faec1676404 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:53.968457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c6f99-cbe7-4015-ad6d-5380e8c3d368 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:53.998079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:54.014377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d608234-7d08-44d8-a6c3-9b1766fee730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:54.015059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d608234-7d08-44d8-a6c3-9b1766fee730 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:54.015338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d608234-7d08-44d8-a6c3-9b1766fee730 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:54.015643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d608234-7d08-44d8-a6c3-9b1766fee730 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:54.016598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d608234-7d08-44d8-a6c3-9b1766fee730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:54.016723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d608234-7d08-44d8-a6c3-9b1766fee730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:54.018040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d608234-7d08-44d8-a6c3-9b1766fee730 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:54.021358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:54.036871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 502/1036] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:54.041883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d608234-7d08-44d8-a6c3-9b1766fee730 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:54.108922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218339a-03cc-4295-9d6e-2faec1676404 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:54.148429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0946fb-7d1f-48f9-9439-3da063869ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:54.149754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0946fb-7d1f-48f9-9439-3da063869ea3 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:54.150195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0946fb-7d1f-48f9-9439-3da063869ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:54.150470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0946fb-7d1f-48f9-9439-3da063869ea3 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:54.151442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0946fb-7d1f-48f9-9439-3da063869ea3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:54.152909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0946fb-7d1f-48f9-9439-3da063869ea3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:54.155630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d608234-7d08-44d8-a6c3-9b1766fee730 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:54.197309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d99857-4ffc-4c25-a919-e1e794fa4a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:54.198121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d99857-4ffc-4c25-a919-e1e794fa4a65 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:54.198395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d99857-4ffc-4c25-a919-e1e794fa4a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:54.198832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d99857-4ffc-4c25-a919-e1e794fa4a65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:54.207441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d99857-4ffc-4c25-a919-e1e794fa4a65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:54.208938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d99857-4ffc-4c25-a919-e1e794fa4a65 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-31d99857-4ffc-4c25-a919-e1e794fa4a65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:54.225381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:54.226496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:54.227489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:54.231674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 503/1037] 10.222.0.64 () {56 vars in 1093 bytes} [Mon Jun 27 09:16:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:54.233251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d99857-4ffc-4c25-a919-e1e794fa4a65 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:54.256188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 535/1038] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:16:53 2022] POST /identity/v3/users => generated 285 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:16:54.315309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:54.334074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8f29d-9c22-4c3b-8a74-bc0bdd1a2148 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:54.334451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8f29d-9c22-4c3b-8a74-bc0bdd1a2148 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:54.334643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8f29d-9c22-4c3b-8a74-bc0bdd1a2148 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:54.336362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 536/1039] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:54.342416 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b877176-c152-401a-8eed-40320b8fae64 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:54.342768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b877176-c152-401a-8eed-40320b8fae64 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:54.343007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b877176-c152-401a-8eed-40320b8fae64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:54.362862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-1b877176-c152-401a-8eed-40320b8fae64 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:54.364164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1b877176-c152-401a-8eed-40320b8fae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:54.388881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d99857-4ffc-4c25-a919-e1e794fa4a65 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:54.443388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ace7b-b9d8-4518-a62c-9e47d4df5d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:54.444417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ace7b-b9d8-4518-a62c-9e47d4df5d16 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:54.444872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ace7b-b9d8-4518-a62c-9e47d4df5d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:54.445300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ace7b-b9d8-4518-a62c-9e47d4df5d16 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:54.446544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ace7b-b9d8-4518-a62c-9e47d4df5d16 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:54.446874 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ace7b-b9d8-4518-a62c-9e47d4df5d16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:54.448433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ace7b-b9d8-4518-a62c-9e47d4df5d16 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:54.463426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 537/1040] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:54.473284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 504/1041] 10.222.0.64 () {56 vars in 1133 bytes} [Mon Jun 27 09:16:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:54.479311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b877176-c152-401a-8eed-40320b8fae64 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:54.511715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ace7b-b9d8-4518-a62c-9e47d4df5d16 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:54.520539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:54.613836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b877176-c152-401a-8eed-40320b8fae64 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:54.654070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38f71c-1b72-419a-8a55-57a81eaad7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:54.654734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38f71c-1b72-419a-8a55-57a81eaad7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:54.655020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38f71c-1b72-419a-8a55-57a81eaad7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:54.655299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38f71c-1b72-419a-8a55-57a81eaad7f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:54.664994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ace7b-b9d8-4518-a62c-9e47d4df5d16 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:54.666846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:54.706254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166285a-6355-4649-805c-176693858543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:54.707074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166285a-6355-4649-805c-176693858543 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:54.707354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166285a-6355-4649-805c-176693858543 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:54.707736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166285a-6355-4649-805c-176693858543 None admin] PATH_INFO: `/v3/services/image` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:54.708407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166285a-6355-4649-805c-176693858543 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:54.709871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166285a-6355-4649-805c-176693858543 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-9166285a-6355-4649-805c-176693858543 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:54.725971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:54.727231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 09:16:54.728306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:54.729751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 505/1042] 10.222.0.64 () {56 vars in 1105 bytes} [Mon Jun 27 09:16:54 2022] GET /identity/v3/services/image => generated 86 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:54.736492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166285a-6355-4649-805c-176693858543 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:54.744042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38f71c-1b72-419a-8a55-57a81eaad7f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:54.745520 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38f71c-1b72-419a-8a55-57a81eaad7f3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:54.785956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:54.856904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166285a-6355-4649-805c-176693858543 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:54.870639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 538/1043] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:54 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 09:16:54.876499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38f71c-1b72-419a-8a55-57a81eaad7f3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:54.898001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c1677-e21d-4eeb-9734-5e06aae3f257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:54.898751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c1677-e21d-4eeb-9734-5e06aae3f257 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:54.899103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c1677-e21d-4eeb-9734-5e06aae3f257 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:54.899452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c1677-e21d-4eeb-9734-5e06aae3f257 None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:54.900274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c1677-e21d-4eeb-9734-5e06aae3f257 None admin] RBAC: Adding query filter params (name=image) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:54.900899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c1677-e21d-4eeb-9734-5e06aae3f257 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:54.902299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c1677-e21d-4eeb-9734-5e06aae3f257 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:54.915604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 506/1044] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:54.922153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c1677-e21d-4eeb-9734-5e06aae3f257 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:54.979365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:55.032950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38f71c-1b72-419a-8a55-57a81eaad7f3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:55.067777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c1677-e21d-4eeb-9734-5e06aae3f257 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:55.069244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:55.076248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470d393-b3cb-403b-9c8a-6093835daf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:55.077057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470d393-b3cb-403b-9c8a-6093835daf40 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:55.077352 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470d393-b3cb-403b-9c8a-6093835daf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:55.077605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470d393-b3cb-403b-9c8a-6093835daf40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:55.089604 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470d393-b3cb-403b-9c8a-6093835daf40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:55.091084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470d393-b3cb-403b-9c8a-6093835daf40 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3470d393-b3cb-403b-9c8a-6093835daf40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:55.111832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:55.113463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:55.121255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:55.121255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:55.121255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:55.121255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:55.121511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22901a62-4285-4f8f-8cb5-223e4eb5708a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:55.121511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22901a62-4285-4f8f-8cb5-223e4eb5708a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:55.121511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22901a62-4285-4f8f-8cb5-223e4eb5708a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:55.121511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22901a62-4285-4f8f-8cb5-223e4eb5708a None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:55.121783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 539/1045] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:55.121852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22901a62-4285-4f8f-8cb5-223e4eb5708a None admin] RBAC: Adding query filter params (type=image) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:55.121852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22901a62-4285-4f8f-8cb5-223e4eb5708a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:55.121852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22901a62-4285-4f8f-8cb5-223e4eb5708a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:55.122046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470d393-b3cb-403b-9c8a-6093835daf40 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:55.141864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 507/1046] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:55.143195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22901a62-4285-4f8f-8cb5-223e4eb5708a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:55.278925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470d393-b3cb-403b-9c8a-6093835daf40 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:55.310704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:55.317783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22901a62-4285-4f8f-8cb5-223e4eb5708a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:55.321139 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e35999-fb7f-4ef5-9543-ac84c0432986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:55.323981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e35999-fb7f-4ef5-9543-ac84c0432986 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:55.323981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e35999-fb7f-4ef5-9543-ac84c0432986 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:55.323981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e35999-fb7f-4ef5-9543-ac84c0432986 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:55.323981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e35999-fb7f-4ef5-9543-ac84c0432986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:55.323981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e35999-fb7f-4ef5-9543-ac84c0432986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:55.324812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e35999-fb7f-4ef5-9543-ac84c0432986 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:55.345253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 540/1047] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:55.346609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e35999-fb7f-4ef5-9543-ac84c0432986 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:55.365807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07190d-7225-47ad-b6b1-43eb1abf538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:55.366483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07190d-7225-47ad-b6b1-43eb1abf538e None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:55.366763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07190d-7225-47ad-b6b1-43eb1abf538e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:55.367032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07190d-7225-47ad-b6b1-43eb1abf538e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:55.367812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07190d-7225-47ad-b6b1-43eb1abf538e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:55.369197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07190d-7225-47ad-b6b1-43eb1abf538e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:55.398804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:55.444812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 508/1048] 10.222.0.64 () {60 vars in 1144 bytes} [Mon Jun 27 09:16:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:16:55.459967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e35999-fb7f-4ef5-9543-ac84c0432986 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:55.519444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655b3b-e716-41e8-b47d-3b10a7baa812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:55.520306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655b3b-e716-41e8-b47d-3b10a7baa812 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:55.520598 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655b3b-e716-41e8-b47d-3b10a7baa812 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:55.520872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655b3b-e716-41e8-b47d-3b10a7baa812 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:55.530208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30655b3b-e716-41e8-b47d-3b10a7baa812 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:55.531737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30655b3b-e716-41e8-b47d-3b10a7baa812 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-30655b3b-e716-41e8-b47d-3b10a7baa812 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:16:55.548910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:55.552846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:55.554017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:16:55.554017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:16:55.554017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:16:55.554017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:16:55.554017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:16:55.554017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:16:55.554017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:16:55.554017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:55.554017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 541/1049] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:55.556303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07190d-7225-47ad-b6b1-43eb1abf538e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:55.718938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07190d-7225-47ad-b6b1-43eb1abf538e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:55.782290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a855f-2398-46e4-97a7-38d9383f9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:55.783378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a855f-2398-46e4-97a7-38d9383f9656 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:55.783895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a855f-2398-46e4-97a7-38d9383f9656 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:55.784381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a855f-2398-46e4-97a7-38d9383f9656 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:55.785842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a855f-2398-46e4-97a7-38d9383f9656 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:55.786382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a855f-2398-46e4-97a7-38d9383f9656 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:55.789112 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a855f-2398-46e4-97a7-38d9383f9656 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:55.818737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 509/1050] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:16:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:55.822336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655b3b-e716-41e8-b47d-3b10a7baa812 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:56.005162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:56.011643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655b3b-e716-41e8-b47d-3b10a7baa812 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:56.074662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab89e7a-c60d-4af8-a003-997e9b60f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:56.075867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab89e7a-c60d-4af8-a003-997e9b60f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:56.076367 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab89e7a-c60d-4af8-a003-997e9b60f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:56.076870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab89e7a-c60d-4af8-a003-997e9b60f5d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:56.079179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab89e7a-c60d-4af8-a003-997e9b60f5d6 None admin] RBAC: Adding query filter params (role.id=e0d6e1d5614e4a4881a0320bda7328cb, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19e0fbe0b5814ad1ae95c65a64b632d6) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:56.079806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab89e7a-c60d-4af8-a003-997e9b60f5d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:56.082695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab89e7a-c60d-4af8-a003-997e9b60f5d6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:56.092943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 542/1051] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:55 2022] GET /identity/v3/role_assignments?user.id=19e0fbe0b5814ad1ae95c65a64b632d6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=e0d6e1d5614e4a4881a0320bda7328cb => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:56.537401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd795c2-55fc-45c6-b9e8-7f17412999e2 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:56.538174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd795c2-55fc-45c6-b9e8-7f17412999e2 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:56.538314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd795c2-55fc-45c6-b9e8-7f17412999e2 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:56.540134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 510/1052] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:16:56.546786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1074844-f966-4ba1-a926-eee2a271fdbc None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:56.547069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1074844-f966-4ba1-a926-eee2a271fdbc None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:56.548034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1074844-f966-4ba1-a926-eee2a271fdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:56.571100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-f1074844-f966-4ba1-a926-eee2a271fdbc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:56.572362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f1074844-f966-4ba1-a926-eee2a271fdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:56.667605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 543/1053] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:16:56.707522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd795c2-55fc-45c6-b9e8-7f17412999e2 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:56.725312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:56.822527 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd795c2-55fc-45c6-b9e8-7f17412999e2 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:56.859239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8f6b3-a336-4131-a68d-c53d3417e4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:56.860652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8f6b3-a336-4131-a68d-c53d3417e4ad None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:56.860924 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8f6b3-a336-4131-a68d-c53d3417e4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:56.861195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8f6b3-a336-4131-a68d-c53d3417e4ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:56.874152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8f6b3-a336-4131-a68d-c53d3417e4ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-5bb8f6b3-a336-4131-a68d-c53d3417e4ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:56.882797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:56.884574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:16:56.885632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:56.887725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 511/1054] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:16:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:56.890454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1074844-f966-4ba1-a926-eee2a271fdbc None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:56.995431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:57.002916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1074844-f966-4ba1-a926-eee2a271fdbc None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:57.039273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b61cd-449f-4e0e-9a21-33d9770da7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:57.039975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b61cd-449f-4e0e-9a21-33d9770da7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:57.040258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b61cd-449f-4e0e-9a21-33d9770da7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:57.040532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b61cd-449f-4e0e-9a21-33d9770da7e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:57.041282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b61cd-449f-4e0e-9a21-33d9770da7e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:57.041584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b61cd-449f-4e0e-9a21-33d9770da7e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:57.042972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b61cd-449f-4e0e-9a21-33d9770da7e5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:57.052065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 544/1055] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:16:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:57.064201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8f6b3-a336-4131-a68d-c53d3417e4ad None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:57.144028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:57.196455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8f6b3-a336-4131-a68d-c53d3417e4ad None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:57.256002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79501aac-5c93-466a-b45d-e888e06db3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:57.257651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79501aac-5c93-466a-b45d-e888e06db3ca None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:57.258140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79501aac-5c93-466a-b45d-e888e06db3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:57.258651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79501aac-5c93-466a-b45d-e888e06db3ca None admin] PATH_INFO: `/v3/users/d3bdbb475a7b4a16a98f767fcaae716b` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:57.287116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79501aac-5c93-466a-b45d-e888e06db3ca None admin] RBAC: Authorizing `identity:get_user(user_id=d3bdbb475a7b4a16a98f767fcaae716b)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:57.288912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79501aac-5c93-466a-b45d-e888e06db3ca None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:57.295911 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 512/1056] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:16:57 2022] GET /identity/v3/users/d3bdbb475a7b4a16a98f767fcaae716b => generated 285 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:57.297989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b61cd-449f-4e0e-9a21-33d9770da7e5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:57.408847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b61cd-449f-4e0e-9a21-33d9770da7e5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:57.446300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51296104-2cbd-4883-8e8f-38c2fd61215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:57.447021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51296104-2cbd-4883-8e8f-38c2fd61215c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:57.447316 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51296104-2cbd-4883-8e8f-38c2fd61215c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:57.447629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51296104-2cbd-4883-8e8f-38c2fd61215c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:57.474673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:57.520832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51296104-2cbd-4883-8e8f-38c2fd61215c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:57.522494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51296104-2cbd-4883-8e8f-38c2fd61215c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:57.654694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 545/1057] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:16:57.660965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79501aac-5c93-466a-b45d-e888e06db3ca None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:57.676893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a74d7-d854-4e96-8f6f-97b188f99e27 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:57.677219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a74d7-d854-4e96-8f6f-97b188f99e27 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:57.677459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a74d7-d854-4e96-8f6f-97b188f99e27 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:57.678987 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 546/1058] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:57.684217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dc1e4-8fef-42be-9e0a-13801433479c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:57.684500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dc1e4-8fef-42be-9e0a-13801433479c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:57.684778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dc1e4-8fef-42be-9e0a-13801433479c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:57.705859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4a2dc1e4-8fef-42be-9e0a-13801433479c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:57.707782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4a2dc1e4-8fef-42be-9e0a-13801433479c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:57.773030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:57.781043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79501aac-5c93-466a-b45d-e888e06db3ca None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:57.795735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 547/1059] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:57.816350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:57.817847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dc1e4-8fef-42be-9e0a-13801433479c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:57.828943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c278d-31a9-4fd6-8190-1981611a5bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:57.830422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c278d-31a9-4fd6-8190-1981611a5bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:57.830925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c278d-31a9-4fd6-8190-1981611a5bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:57.831439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c278d-31a9-4fd6-8190-1981611a5bf2 None admin] PATH_INFO: `/v3/projects/f51d580536ff42a3a157af7405926881` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:57.833654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c278d-31a9-4fd6-8190-1981611a5bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=f51d580536ff42a3a157af7405926881)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:57.836302 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c278d-31a9-4fd6-8190-1981611a5bf2 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:57.844917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 513/1060] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:16:57 2022] GET /identity/v3/projects/f51d580536ff42a3a157af7405926881 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:57.846969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c278d-31a9-4fd6-8190-1981611a5bf2 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:57.959433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dc1e4-8fef-42be-9e0a-13801433479c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:57.971259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c278d-31a9-4fd6-8190-1981611a5bf2 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:58.002564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27825c18-bb54-4acd-8d94-47dec4ef9b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:58.003462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825c18-bb54-4acd-8d94-47dec4ef9b56 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:58.004009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825c18-bb54-4acd-8d94-47dec4ef9b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:58.004518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825c18-bb54-4acd-8d94-47dec4ef9b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:58.011017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a56eb8-e60a-4167-88af-5b4c2634f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:58.012005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a56eb8-e60a-4167-88af-5b4c2634f59b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:58.012284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a56eb8-e60a-4167-88af-5b4c2634f59b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:58.012395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a56eb8-e60a-4167-88af-5b4c2634f59b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:58.013205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a56eb8-e60a-4167-88af-5b4c2634f59b None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=f51d580536ff42a3a157af7405926881, user.id=d3bdbb475a7b4a16a98f767fcaae716b) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:58.013521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a56eb8-e60a-4167-88af-5b4c2634f59b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:58.014975 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a56eb8-e60a-4167-88af-5b4c2634f59b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:58.022240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 514/1061] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:16:57 2022] GET /identity/v3/role_assignments?user.id=d3bdbb475a7b4a16a98f767fcaae716b&scope.project.id=f51d580536ff42a3a157af7405926881&role.id=03b724e03f704f2ab338b64025a3db6d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:58.121256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:58.148322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27825c18-bb54-4acd-8d94-47dec4ef9b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:58.151514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27825c18-bb54-4acd-8d94-47dec4ef9b56 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:58.194341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c3bbd-d803-4006-95b1-bd191d4b2b80 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:58.198489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c3bbd-d803-4006-95b1-bd191d4b2b80 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:58.198489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c3bbd-d803-4006-95b1-bd191d4b2b80 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:58.198489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 515/1062] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:16:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:16:58.205741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a4a9f-da98-42cb-a179-5c4e76629783 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:58.206214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a4a9f-da98-42cb-a179-5c4e76629783 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:58.206681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a4a9f-da98-42cb-a179-5c4e76629783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:58.227581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e22a4a9f-da98-42cb-a179-5c4e76629783 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:16:58.228701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e22a4a9f-da98-42cb-a179-5c4e76629783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:16:58.295562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:58.297144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 548/1063] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:16:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:16:58.303193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27825c18-bb54-4acd-8d94-47dec4ef9b56 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:58.308161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 516/1064] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:16:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:16:58.331620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a4a9f-da98-42cb-a179-5c4e76629783 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:58.432074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:58.459971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a4a9f-da98-42cb-a179-5c4e76629783 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:58.462159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27825c18-bb54-4acd-8d94-47dec4ef9b56 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:58.502231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68aad7d-b6e2-4266-b44c-fa91df9b66a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:58.503379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68aad7d-b6e2-4266-b44c-fa91df9b66a0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:58.503836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68aad7d-b6e2-4266-b44c-fa91df9b66a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:58.504253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68aad7d-b6e2-4266-b44c-fa91df9b66a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:58.520607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68aad7d-b6e2-4266-b44c-fa91df9b66a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:16:58.521734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00768814-0362-4ae8-af02-da39fca6e7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:58.522544 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00768814-0362-4ae8-af02-da39fca6e7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:58.522820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00768814-0362-4ae8-af02-da39fca6e7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:58.523083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00768814-0362-4ae8-af02-da39fca6e7e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-b68aad7d-b6e2-4266-b44c-fa91df9b66a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:16:58.529788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:58.530891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:58.532030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 517/1065] 10.222.0.64 () {56 vars in 1120 bytes} [Mon Jun 27 09:16:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:58.537513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68aad7d-b6e2-4266-b44c-fa91df9b66a0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:58.537793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00768814-0362-4ae8-af02-da39fca6e7e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:58.539230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00768814-0362-4ae8-af02-da39fca6e7e5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-00768814-0362-4ae8-af02-da39fca6e7e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:58.560338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:58.561338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:58.562239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:58.562239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:58.562239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:58.562239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:16:58.562239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 549/1066] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:58.568470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00768814-0362-4ae8-af02-da39fca6e7e5 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:58.595287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:58.649733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68aad7d-b6e2-4266-b44c-fa91df9b66a0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:58.702772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10e37d-45cd-4ff7-bdd9-beffb5b4bd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:58.704548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00768814-0362-4ae8-af02-da39fca6e7e5 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:58.705855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10e37d-45cd-4ff7-bdd9-beffb5b4bd9e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:58.706141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10e37d-45cd-4ff7-bdd9-beffb5b4bd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:58.706417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10e37d-45cd-4ff7-bdd9-beffb5b4bd9e None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:58.707266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10e37d-45cd-4ff7-bdd9-beffb5b4bd9e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:58.707592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10e37d-45cd-4ff7-bdd9-beffb5b4bd9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:58.709001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10e37d-45cd-4ff7-bdd9-beffb5b4bd9e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:58.718614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 518/1067] 10.222.0.64 () {56 vars in 1115 bytes} [Mon Jun 27 09:16:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:16:58.725050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10e37d-45cd-4ff7-bdd9-beffb5b4bd9e None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:58.765753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954e6a8-d46d-4460-b83d-9be760d657f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:58.767407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954e6a8-d46d-4460-b83d-9be760d657f6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:58.767943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954e6a8-d46d-4460-b83d-9be760d657f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:58.768440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954e6a8-d46d-4460-b83d-9be760d657f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:58.769779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954e6a8-d46d-4460-b83d-9be760d657f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:58.770274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954e6a8-d46d-4460-b83d-9be760d657f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:58.772891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954e6a8-d46d-4460-b83d-9be760d657f6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:58.786333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:58.807564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 550/1068] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:58.888172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10e37d-45cd-4ff7-bdd9-beffb5b4bd9e None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:58.904821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:58.942426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dde66-4939-412f-b8aa-b4b12c0b13d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:58.943997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dde66-4939-412f-b8aa-b4b12c0b13d9 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:58.944621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dde66-4939-412f-b8aa-b4b12c0b13d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:58.945141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dde66-4939-412f-b8aa-b4b12c0b13d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:58.958951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dde66-4939-412f-b8aa-b4b12c0b13d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:58.960567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dde66-4939-412f-b8aa-b4b12c0b13d9 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-fc1dde66-4939-412f-b8aa-b4b12c0b13d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:58.979719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:58.981004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:58.982504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:58.982504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:58.982504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:58.982504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:58.982504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 519/1069] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:16:58.987773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954e6a8-d46d-4460-b83d-9be760d657f6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:59.132724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954e6a8-d46d-4460-b83d-9be760d657f6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:59.172153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc2830-fe48-4c35-ba3c-1120ef1e2cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:59.173622 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc2830-fe48-4c35-ba3c-1120ef1e2cba None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:59.173908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc2830-fe48-4c35-ba3c-1120ef1e2cba None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:59.174186 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc2830-fe48-4c35-ba3c-1120ef1e2cba None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:59.174941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc2830-fe48-4c35-ba3c-1120ef1e2cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:59.175335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc2830-fe48-4c35-ba3c-1120ef1e2cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:59.177742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc2830-fe48-4c35-ba3c-1120ef1e2cba None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:59.192424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 551/1070] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:59.198483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dde66-4939-412f-b8aa-b4b12c0b13d9 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:59.320199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dde66-4939-412f-b8aa-b4b12c0b13d9 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:59.357808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137ee5a-4484-4329-953b-ee97acc1551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:59.358603 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137ee5a-4484-4329-953b-ee97acc1551d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:59.358882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137ee5a-4484-4329-953b-ee97acc1551d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:59.359157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137ee5a-4484-4329-953b-ee97acc1551d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:59.368518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137ee5a-4484-4329-953b-ee97acc1551d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:59.369974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137ee5a-4484-4329-953b-ee97acc1551d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-2137ee5a-4484-4329-953b-ee97acc1551d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:16:59.385030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:59.386164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:59.387106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 520/1071] 10.222.0.64 () {56 vars in 1093 bytes} [Mon Jun 27 09:16:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:59.394446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc2830-fe48-4c35-ba3c-1120ef1e2cba None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:59.472650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:59.517364 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc2830-fe48-4c35-ba3c-1120ef1e2cba None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:59.557924 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63631f0a-df5b-452d-a40e-72068fb0cb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:59.558696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63631f0a-df5b-452d-a40e-72068fb0cb27 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:59.559052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63631f0a-df5b-452d-a40e-72068fb0cb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:59.559400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63631f0a-df5b-452d-a40e-72068fb0cb27 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:59.560373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63631f0a-df5b-452d-a40e-72068fb0cb27 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:59.560806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63631f0a-df5b-452d-a40e-72068fb0cb27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:59.562531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63631f0a-df5b-452d-a40e-72068fb0cb27 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:59.585428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 552/1072] 10.222.0.64 () {56 vars in 1133 bytes} [Mon Jun 27 09:16:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:59.592200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137ee5a-4484-4329-953b-ee97acc1551d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:59.623113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:59.709676 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137ee5a-4484-4329-953b-ee97acc1551d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:59.760044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4220f-9acb-4f36-b0e9-64417ab5cf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:59.761316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4220f-9acb-4f36-b0e9-64417ab5cf02 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:59.761805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4220f-9acb-4f36-b0e9-64417ab5cf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:59.762218 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4220f-9acb-4f36-b0e9-64417ab5cf02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:59.778568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4220f-9acb-4f36-b0e9-64417ab5cf02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:59.780087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4220f-9acb-4f36-b0e9-64417ab5cf02 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-bfb4220f-9acb-4f36-b0e9-64417ab5cf02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:16:59.799848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:16:59.800960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:16:59.801862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:16:59.801862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:16:59.801862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:16:59.801862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:16:59.801862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 521/1073] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:16:59.807259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63631f0a-df5b-452d-a40e-72068fb0cb27 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:16:59.932390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63631f0a-df5b-452d-a40e-72068fb0cb27 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:16:59.950851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:16:59.972047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3be9ed-0777-40de-9bd9-7e3a55575e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:16:59.972047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3be9ed-0777-40de-9bd9-7e3a55575e8e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:16:59.972047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3be9ed-0777-40de-9bd9-7e3a55575e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:16:59.972047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3be9ed-0777-40de-9bd9-7e3a55575e8e None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:16:59.972434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3be9ed-0777-40de-9bd9-7e3a55575e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:16:59.972434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3be9ed-0777-40de-9bd9-7e3a55575e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:16:59.973600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3be9ed-0777-40de-9bd9-7e3a55575e8e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:16:59.985769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 553/1074] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:16:59.991541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4220f-9acb-4f36-b0e9-64417ab5cf02 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:00.070044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:00.092661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4220f-9acb-4f36-b0e9-64417ab5cf02 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:00.130856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a2bf5-40d3-41fa-a9a5-500bbde7c13b None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:00.131318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a2bf5-40d3-41fa-a9a5-500bbde7c13b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:00.131471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a2bf5-40d3-41fa-a9a5-500bbde7c13b None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:00.133041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 554/1075] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:17:00.134042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a880f-9de5-4cc1-8481-423b0addacb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:00.134864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a880f-9de5-4cc1-8481-423b0addacb3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:00.135150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a880f-9de5-4cc1-8481-423b0addacb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:00.135426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a880f-9de5-4cc1-8481-423b0addacb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:00.138459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc307d-cbc0-4ec0-bae0-fc0ad158fc3c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:00.138758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc307d-cbc0-4ec0-bae0-fc0ad158fc3c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:00.139041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc307d-cbc0-4ec0-bae0-fc0ad158fc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:00.144490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a880f-9de5-4cc1-8481-423b0addacb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:00.146339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a880f-9de5-4cc1-8481-423b0addacb3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:00.158089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c9cc307d-cbc0-4ec0-bae0-fc0ad158fc3c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:00.159312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c9cc307d-cbc0-4ec0-bae0-fc0ad158fc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-912a880f-9de5-4cc1-8481-423b0addacb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:17:00.161461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:00.162661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:00.163766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:17:00.163766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:17:00.163766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:17:00.163766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:17:00.163766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:17:00.163766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:17:00.163766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:17:00.163766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:17:00.163766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 522/1076] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:16:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:17:00.168629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a880f-9de5-4cc1-8481-423b0addacb3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:00.237263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 555/1077] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:17:00.260349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc307d-cbc0-4ec0-bae0-fc0ad158fc3c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:00.279597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a880f-9de5-4cc1-8481-423b0addacb3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:00.316973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049f314-bc78-43f1-8645-27bc26e2288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:00.317644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049f314-bc78-43f1-8645-27bc26e2288b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:00.317927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049f314-bc78-43f1-8645-27bc26e2288b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:00.318231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049f314-bc78-43f1-8645-27bc26e2288b None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:00.319004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049f314-bc78-43f1-8645-27bc26e2288b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:00.319314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049f314-bc78-43f1-8645-27bc26e2288b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:00.320825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049f314-bc78-43f1-8645-27bc26e2288b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:00.323649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:00.333951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 523/1078] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:17:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:00.340616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049f314-bc78-43f1-8645-27bc26e2288b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:00.372272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc307d-cbc0-4ec0-bae0-fc0ad158fc3c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:00.380653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:00.408774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b526ae-7d93-4fea-a7aa-15b19bac760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:00.409585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b526ae-7d93-4fea-a7aa-15b19bac760f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:00.409869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b526ae-7d93-4fea-a7aa-15b19bac760f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:00.410146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b526ae-7d93-4fea-a7aa-15b19bac760f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:00.421378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b526ae-7d93-4fea-a7aa-15b19bac760f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-41b526ae-7d93-4fea-a7aa-15b19bac760f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:17:00.429807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:00.430982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:00.432168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 556/1079] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:17:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:17:00.437560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b526ae-7d93-4fea-a7aa-15b19bac760f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:00.449457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049f314-bc78-43f1-8645-27bc26e2288b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:00.487369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4184de-e37d-4e69-838e-bd93f8cb1a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:00.488207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4184de-e37d-4e69-838e-bd93f8cb1a04 None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:00.488489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4184de-e37d-4e69-838e-bd93f8cb1a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:00.488759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4184de-e37d-4e69-838e-bd93f8cb1a04 None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19e0fbe0b5814ad1ae95c65a64b632d6/roles/e0d6e1d5614e4a4881a0320bda7328cb` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:00.491068 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4184de-e37d-4e69-838e-bd93f8cb1a04 None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=19e0fbe0b5814ad1ae95c65a64b632d6, role_id=e0d6e1d5614e4a4881a0320bda7328cb)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:00.492621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4184de-e37d-4e69-838e-bd93f8cb1a04 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:00.551490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b526ae-7d93-4fea-a7aa-15b19bac760f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:00.551822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 524/1080] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:17:00 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19e0fbe0b5814ad1ae95c65a64b632d6/roles/e0d6e1d5614e4a4881a0320bda7328cb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:17:00.613425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f6538-292e-440c-a1fd-3e2f959379d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:00.614117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f6538-292e-440c-a1fd-3e2f959379d6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:00.614504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f6538-292e-440c-a1fd-3e2f959379d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:00.615733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f6538-292e-440c-a1fd-3e2f959379d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:00.615733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f6538-292e-440c-a1fd-3e2f959379d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:00.615733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f6538-292e-440c-a1fd-3e2f959379d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:00.617695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f6538-292e-440c-a1fd-3e2f959379d6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:00.628621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 557/1081] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:17:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:00.634857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4184de-e37d-4e69-838e-bd93f8cb1a04 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:00.705557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:00.751098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4184de-e37d-4e69-838e-bd93f8cb1a04 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:00.793095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff595-1011-4d76-8473-14f8e3f31dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:00.794022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff595-1011-4d76-8473-14f8e3f31dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:00.794304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff595-1011-4d76-8473-14f8e3f31dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:00.794581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ff595-1011-4d76-8473-14f8e3f31dd5 None admin] PATH_INFO: `/v3/users/d3bdbb475a7b4a16a98f767fcaae716b` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:00.795938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ff595-1011-4d76-8473-14f8e3f31dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=d3bdbb475a7b4a16a98f767fcaae716b)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:00.797982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ff595-1011-4d76-8473-14f8e3f31dd5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:00.799798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 525/1082] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:00 2022] GET /identity/v3/users/d3bdbb475a7b4a16a98f767fcaae716b => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:00.805850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f6538-292e-440c-a1fd-3e2f959379d6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:00.853462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:00.937114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f6538-292e-440c-a1fd-3e2f959379d6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:00.973120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ed127-c89a-4c08-ae50-9dfe530c9da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:00.973998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ed127-c89a-4c08-ae50-9dfe530c9da2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:00.974269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ed127-c89a-4c08-ae50-9dfe530c9da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:00.974539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ed127-c89a-4c08-ae50-9dfe530c9da2 None admin] PATH_INFO: `/v3/projects/f51d580536ff42a3a157af7405926881` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:00.975834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ed127-c89a-4c08-ae50-9dfe530c9da2 None admin] RBAC: Authorizing `identity:get_project(project_id=f51d580536ff42a3a157af7405926881)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:00.977279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ed127-c89a-4c08-ae50-9dfe530c9da2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:00.979019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 558/1083] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:00 2022] GET /identity/v3/projects/f51d580536ff42a3a157af7405926881 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:00.985750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff595-1011-4d76-8473-14f8e3f31dd5 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:01.086366 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ff595-1011-4d76-8473-14f8e3f31dd5 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:01.110445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:01.121163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8098ab5-50bd-4099-9ae7-8826eba17e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:01.122046 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8098ab5-50bd-4099-9ae7-8826eba17e94 None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:01.122323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8098ab5-50bd-4099-9ae7-8826eba17e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:01.122595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8098ab5-50bd-4099-9ae7-8826eba17e94 None admin] PATH_INFO: `/v3/projects/f51d580536ff42a3a157af7405926881/users/d3bdbb475a7b4a16a98f767fcaae716b/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:01.128184 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8098ab5-50bd-4099-9ae7-8826eba17e94 None admin] RBAC: Authorizing `identity:create_grant(project_id=f51d580536ff42a3a157af7405926881, user_id=d3bdbb475a7b4a16a98f767fcaae716b, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:01.128184 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8098ab5-50bd-4099-9ae7-8826eba17e94 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:01.248832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 526/1084] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:17:00 2022] PUT /identity/v3/projects/f51d580536ff42a3a157af7405926881/users/d3bdbb475a7b4a16a98f767fcaae716b/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:17:02.639303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8428cc3-3796-46df-8460-77614e88ce89 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:02.639834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8428cc3-3796-46df-8460-77614e88ce89 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:02.640706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8428cc3-3796-46df-8460-77614e88ce89 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:02.642767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 559/1085] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:02.649345 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786e96-3bad-4bf8-b67c-b62da960a4be None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:02.649888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786e96-3bad-4bf8-b67c-b62da960a4be None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:02.650465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7786e96-3bad-4bf8-b67c-b62da960a4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:02.670008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f7786e96-3bad-4bf8-b67c-b62da960a4be None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:02.671256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f7786e96-3bad-4bf8-b67c-b62da960a4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:02.748573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 527/1086] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:02.772744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8428cc3-3796-46df-8460-77614e88ce89 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:02.882998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:02.899299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8428cc3-3796-46df-8460-77614e88ce89 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:02.933985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01a14a-b71b-4438-b444-5b7c6fc8ad09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:02.934808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01a14a-b71b-4438-b444-5b7c6fc8ad09 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:02.935090 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01a14a-b71b-4438-b444-5b7c6fc8ad09 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:02.935366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01a14a-b71b-4438-b444-5b7c6fc8ad09 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:02.946911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01a14a-b71b-4438-b444-5b7c6fc8ad09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-8d01a14a-b71b-4438-b444-5b7c6fc8ad09 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:17:02.955889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:02.956859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:02.957704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 560/1087] 10.222.0.64 () {56 vars in 1120 bytes} [Mon Jun 27 09:17:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:02.963054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7786e96-3bad-4bf8-b67c-b62da960a4be None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:03.073486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7786e96-3bad-4bf8-b67c-b62da960a4be None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:03.091675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:03.116566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35783271-657c-43a8-a6dd-2c64c6761d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:03.117341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35783271-657c-43a8-a6dd-2c64c6761d94 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.117626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35783271-657c-43a8-a6dd-2c64c6761d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.117908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35783271-657c-43a8-a6dd-2c64c6761d94 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.118676 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35783271-657c-43a8-a6dd-2c64c6761d94 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:03.118987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35783271-657c-43a8-a6dd-2c64c6761d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:03.120419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35783271-657c-43a8-a6dd-2c64c6761d94 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:03.135794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 528/1088] 10.222.0.64 () {56 vars in 1115 bytes} [Mon Jun 27 09:17:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:03.137554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01a14a-b71b-4438-b444-5b7c6fc8ad09 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:03.316653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01a14a-b71b-4438-b444-5b7c6fc8ad09 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:03.331452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9fb15-5c41-4812-b8cc-8dfb11033b12 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.331817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9fb15-5c41-4812-b8cc-8dfb11033b12 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.332092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9fb15-5c41-4812-b8cc-8dfb11033b12 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.333513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 529/1089] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:17:03.341746 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4428fde-2ed2-492e-8337-62a434bca700 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.342035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4428fde-2ed2-492e-8337-62a434bca700 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.342321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4428fde-2ed2-492e-8337-62a434bca700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.357265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76124c34-6798-4fd2-a562-c28965130426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:03.358397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76124c34-6798-4fd2-a562-c28965130426 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.358784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76124c34-6798-4fd2-a562-c28965130426 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.359149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76124c34-6798-4fd2-a562-c28965130426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.362005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f4428fde-2ed2-492e-8337-62a434bca700 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:03.363395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f4428fde-2ed2-492e-8337-62a434bca700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:03.377431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76124c34-6798-4fd2-a562-c28965130426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:03.378971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76124c34-6798-4fd2-a562-c28965130426 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-76124c34-6798-4fd2-a562-c28965130426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:17:03.399131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:03.400227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:03.402588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:17:03.402588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:17:03.402588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:17:03.402588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:03.402588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 561/1090] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:03.407253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76124c34-6798-4fd2-a562-c28965130426 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:03.420129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:03.444088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 530/1091] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:17:03.469899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4428fde-2ed2-492e-8337-62a434bca700 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:03.526693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76124c34-6798-4fd2-a562-c28965130426 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:03.564039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077a96d-8a0b-4b56-a7b7-4b545dc444f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:03.564713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077a96d-8a0b-4b56-a7b7-4b545dc444f7 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.564993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077a96d-8a0b-4b56-a7b7-4b545dc444f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.565263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077a96d-8a0b-4b56-a7b7-4b545dc444f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.566011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077a96d-8a0b-4b56-a7b7-4b545dc444f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:03.566426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077a96d-8a0b-4b56-a7b7-4b545dc444f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:03.567975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077a96d-8a0b-4b56-a7b7-4b545dc444f7 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:03.580840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 562/1092] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:03.586768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077a96d-8a0b-4b56-a7b7-4b545dc444f7 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:03.592908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4428fde-2ed2-492e-8337-62a434bca700 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:03.602268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:03.632605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f02d7-b22d-40d4-bbff-bd9a8c0f6e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:03.633405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f02d7-b22d-40d4-bbff-bd9a8c0f6e4c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.633686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f02d7-b22d-40d4-bbff-bd9a8c0f6e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.633959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f02d7-b22d-40d4-bbff-bd9a8c0f6e4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.644936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f02d7-b22d-40d4-bbff-bd9a8c0f6e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-b71f02d7-b22d-40d4-bbff-bd9a8c0f6e4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:17:03.653784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:03.655337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:03.656357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:17:03.659715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 531/1093] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:17:03.661852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f02d7-b22d-40d4-bbff-bd9a8c0f6e4c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:03.670891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:03.707476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077a96d-8a0b-4b56-a7b7-4b545dc444f7 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:03.744601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b4b26-9d4a-496f-88b1-786e0fa5ab61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:03.745410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b4b26-9d4a-496f-88b1-786e0fa5ab61 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.745690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b4b26-9d4a-496f-88b1-786e0fa5ab61 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.745964 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b4b26-9d4a-496f-88b1-786e0fa5ab61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.757957 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b4b26-9d4a-496f-88b1-786e0fa5ab61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:03.759425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b4b26-9d4a-496f-88b1-786e0fa5ab61 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:03.766322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f02d7-b22d-40d4-bbff-bd9a8c0f6e4c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-949b4b26-9d4a-496f-88b1-786e0fa5ab61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:17:03.774664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:03.775969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:03.777256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 563/1094] 10.222.0.64 () {56 vars in 1093 bytes} [Mon Jun 27 09:17:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:17:03.782471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b4b26-9d4a-496f-88b1-786e0fa5ab61 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:03.803953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50b0fa-e754-4707-b6f4-62a114618196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:03.804624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50b0fa-e754-4707-b6f4-62a114618196 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.804910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50b0fa-e754-4707-b6f4-62a114618196 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.805191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50b0fa-e754-4707-b6f4-62a114618196 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.805956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50b0fa-e754-4707-b6f4-62a114618196 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:03.806286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50b0fa-e754-4707-b6f4-62a114618196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:03.807685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50b0fa-e754-4707-b6f4-62a114618196 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:03.816272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 532/1095] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:17:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:03.822215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50b0fa-e754-4707-b6f4-62a114618196 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:03.853417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:03.891281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b4b26-9d4a-496f-88b1-786e0fa5ab61 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:03.919151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:03.927659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea28cb-aad6-4c97-9f6b-56bacea06cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:03.928350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea28cb-aad6-4c97-9f6b-56bacea06cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.928638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea28cb-aad6-4c97-9f6b-56bacea06cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.929337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea28cb-aad6-4c97-9f6b-56bacea06cc0 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.929760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea28cb-aad6-4c97-9f6b-56bacea06cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:03.930076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea28cb-aad6-4c97-9f6b-56bacea06cc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:03.931762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea28cb-aad6-4c97-9f6b-56bacea06cc0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:03.940529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50b0fa-e754-4707-b6f4-62a114618196 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:03.949774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 564/1096] 10.222.0.64 () {56 vars in 1133 bytes} [Mon Jun 27 09:17:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:03.955429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea28cb-aad6-4c97-9f6b-56bacea06cc0 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:03.988782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d89577-bc85-426b-8598-e33f6ea855bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:03.989664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d89577-bc85-426b-8598-e33f6ea855bd None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:03.990008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d89577-bc85-426b-8598-e33f6ea855bd None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:03.990291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d89577-bc85-426b-8598-e33f6ea855bd None admin] PATH_INFO: `/v3/users/d3bdbb475a7b4a16a98f767fcaae716b` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:03.991790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d89577-bc85-426b-8598-e33f6ea855bd None admin] RBAC: Authorizing `identity:get_user(user_id=d3bdbb475a7b4a16a98f767fcaae716b)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:03.993278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d89577-bc85-426b-8598-e33f6ea855bd None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:03.995178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 533/1097] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:03 2022] GET /identity/v3/users/d3bdbb475a7b4a16a98f767fcaae716b => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:04.001202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d89577-bc85-426b-8598-e33f6ea855bd None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:04.110729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea28cb-aad6-4c97-9f6b-56bacea06cc0 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:04.111757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:04.118450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d89577-bc85-426b-8598-e33f6ea855bd None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:04.160369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fc3dc-dd1e-4e3b-869b-09476b154c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:04.161381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fc3dc-dd1e-4e3b-869b-09476b154c61 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:04.161828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fc3dc-dd1e-4e3b-869b-09476b154c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:04.162239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fc3dc-dd1e-4e3b-869b-09476b154c61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:04.173389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf852e-832e-4698-8314-28a76ddf14c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:04.174418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf852e-832e-4698-8314-28a76ddf14c3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:04.174876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf852e-832e-4698-8314-28a76ddf14c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:04.175315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf852e-832e-4698-8314-28a76ddf14c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:04.242627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:04.260716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8fc3dc-dd1e-4e3b-869b-09476b154c61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:04.262216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8fc3dc-dd1e-4e3b-869b-09476b154c61 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:04.263825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf852e-832e-4698-8314-28a76ddf14c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:04.265311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf852e-832e-4698-8314-28a76ddf14c3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:04.377305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:04.384244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 534/1098] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:17:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 09:17:04.390121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf852e-832e-4698-8314-28a76ddf14c3 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:04.421235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 565/1099] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:17:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 09:17:04.427825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fc3dc-dd1e-4e3b-869b-09476b154c61 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:04.515183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf852e-832e-4698-8314-28a76ddf14c3 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:04.534871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:04.556248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e06f6-f5b3-4770-9d6d-01416f3b76f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:04.557124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e06f6-f5b3-4770-9d6d-01416f3b76f1 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:04.557403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e06f6-f5b3-4770-9d6d-01416f3b76f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:04.557675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e06f6-f5b3-4770-9d6d-01416f3b76f1 None admin] PATH_INFO: `/v3/projects/f51d580536ff42a3a157af7405926881` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:04.558965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e06f6-f5b3-4770-9d6d-01416f3b76f1 None admin] RBAC: Authorizing `identity:get_project(project_id=f51d580536ff42a3a157af7405926881)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:04.560458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e06f6-f5b3-4770-9d6d-01416f3b76f1 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:04.561835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fc3dc-dd1e-4e3b-869b-09476b154c61 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:04.562471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 535/1100] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:04 2022] GET /identity/v3/projects/f51d580536ff42a3a157af7405926881 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:04.573175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e06f6-f5b3-4770-9d6d-01416f3b76f1 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:04.602241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52bff9-d348-42d2-85c1-d10bd09f397b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:04.603051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52bff9-d348-42d2-85c1-d10bd09f397b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:04.603334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52bff9-d348-42d2-85c1-d10bd09f397b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:04.603640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52bff9-d348-42d2-85c1-d10bd09f397b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:04.615747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52bff9-d348-42d2-85c1-d10bd09f397b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:04.617398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52bff9-d348-42d2-85c1-d10bd09f397b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-9e52bff9-d348-42d2-85c1-d10bd09f397b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:17:04.638272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:04.640175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:04.643014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:17:04.643014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:17:04.643014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:17:04.643014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:04.643014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 566/1101] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:17:04.645963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52bff9-d348-42d2-85c1-d10bd09f397b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:04.676380 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:04.734273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e06f6-f5b3-4770-9d6d-01416f3b76f1 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:04.760630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52bff9-d348-42d2-85c1-d10bd09f397b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:04.772772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673f180-45dd-4582-a401-229f6e92ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:04.773427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673f180-45dd-4582-a401-229f6e92ed32 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:04.773783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673f180-45dd-4582-a401-229f6e92ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:04.773980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673f180-45dd-4582-a401-229f6e92ed32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:04.774815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673f180-45dd-4582-a401-229f6e92ed32 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=f51d580536ff42a3a157af7405926881, user.id=d3bdbb475a7b4a16a98f767fcaae716b) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:04.775130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673f180-45dd-4582-a401-229f6e92ed32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:04.776903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673f180-45dd-4582-a401-229f6e92ed32 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:04.783597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 536/1102] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:17:04 2022] GET /identity/v3/role_assignments?user.id=d3bdbb475a7b4a16a98f767fcaae716b&scope.project.id=f51d580536ff42a3a157af7405926881&role.id=03b724e03f704f2ab338b64025a3db6d => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:04.804471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdcfd8-1c50-4e97-a9fe-328916d3a5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:04.805194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdcfd8-1c50-4e97-a9fe-328916d3a5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:04.805499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdcfd8-1c50-4e97-a9fe-328916d3a5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:04.805798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdcfd8-1c50-4e97-a9fe-328916d3a5d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:04.806775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdcfd8-1c50-4e97-a9fe-328916d3a5d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:04.807167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdcfd8-1c50-4e97-a9fe-328916d3a5d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:04.808975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdcfd8-1c50-4e97-a9fe-328916d3a5d1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:04.818829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:04.825725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 567/1103] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:04.832944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673f180-45dd-4582-a401-229f6e92ed32 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:04.993770 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673f180-45dd-4582-a401-229f6e92ed32 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:05.025119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:05.037518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818808f6-d5a2-43b2-9e97-74e9179d359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:05.038386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818808f6-d5a2-43b2-9e97-74e9179d359c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:05.038702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818808f6-d5a2-43b2-9e97-74e9179d359c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:05.039014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818808f6-d5a2-43b2-9e97-74e9179d359c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:05.048243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818808f6-d5a2-43b2-9e97-74e9179d359c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:05.049689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818808f6-d5a2-43b2-9e97-74e9179d359c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-818808f6-d5a2-43b2-9e97-74e9179d359c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:17:05.066475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:05.067654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:05.068682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:17:05.068682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:17:05.068682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:17:05.068682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:17:05.068682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:17:05.068682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:17:05.068682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:17:05.068682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:17:05.068682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 537/1104] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:17:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:05.075499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdcfd8-1c50-4e97-a9fe-328916d3a5d1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:05.195903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdcfd8-1c50-4e97-a9fe-328916d3a5d1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:05.243397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9bf77-8425-4332-8b11-8de55db2c057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:05.244679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9bf77-8425-4332-8b11-8de55db2c057 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:05.245074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9bf77-8425-4332-8b11-8de55db2c057 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:05.245586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9bf77-8425-4332-8b11-8de55db2c057 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:05.247274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9bf77-8425-4332-8b11-8de55db2c057 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:05.250287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9bf77-8425-4332-8b11-8de55db2c057 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:05.253325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9bf77-8425-4332-8b11-8de55db2c057 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:05.276956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 568/1105] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:17:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:05.283431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818808f6-d5a2-43b2-9e97-74e9179d359c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:05.401611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818808f6-d5a2-43b2-9e97-74e9179d359c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:05.449929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553f442-118a-4ccb-8562-451d211f4656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:05.451054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553f442-118a-4ccb-8562-451d211f4656 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:05.451431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553f442-118a-4ccb-8562-451d211f4656 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:05.451849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553f442-118a-4ccb-8562-451d211f4656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:05.453124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553f442-118a-4ccb-8562-451d211f4656 None admin] RBAC: Adding query filter params (role.id=e0d6e1d5614e4a4881a0320bda7328cb, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19e0fbe0b5814ad1ae95c65a64b632d6) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:05.453675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553f442-118a-4ccb-8562-451d211f4656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:05.455352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553f442-118a-4ccb-8562-451d211f4656 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:05.462115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 538/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:17:05 2022] GET /identity/v3/role_assignments?user.id=19e0fbe0b5814ad1ae95c65a64b632d6&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=e0d6e1d5614e4a4881a0320bda7328cb => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:07.019369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec234cc-7c37-477a-83fa-e35d7b1b8871 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:07.019971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec234cc-7c37-477a-83fa-e35d7b1b8871 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:07.019971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec234cc-7c37-477a-83fa-e35d7b1b8871 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:07.021396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 569/1107] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:17:07.027641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34214df-d630-4c01-98bd-eb503177587a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:07.028183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34214df-d630-4c01-98bd-eb503177587a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:07.028183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34214df-d630-4c01-98bd-eb503177587a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:07.046258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b34214df-d630-4c01-98bd-eb503177587a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:07.048105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b34214df-d630-4c01-98bd-eb503177587a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:07.098720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:07.119127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 539/1108] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:07.145705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec234cc-7c37-477a-83fa-e35d7b1b8871 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:07.210289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:07.253629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec234cc-7c37-477a-83fa-e35d7b1b8871 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:07.294826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdc5fc-95ec-417e-9b9e-2ecd182e7490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:07.295664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdc5fc-95ec-417e-9b9e-2ecd182e7490 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:07.295945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdc5fc-95ec-417e-9b9e-2ecd182e7490 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:07.296220 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdc5fc-95ec-417e-9b9e-2ecd182e7490 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:07.297501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fdc5fc-95ec-417e-9b9e-2ecd182e7490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:07.298945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fdc5fc-95ec-417e-9b9e-2ecd182e7490 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:07.300627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 570/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:17:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:07.306353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34214df-d630-4c01-98bd-eb503177587a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:07.413048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34214df-d630-4c01-98bd-eb503177587a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:07.453097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07019b76-3bb2-4200-9f16-e4a64f1e10a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:07.454363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07019b76-3bb2-4200-9f16-e4a64f1e10a9 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:07.454645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07019b76-3bb2-4200-9f16-e4a64f1e10a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:07.454918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07019b76-3bb2-4200-9f16-e4a64f1e10a9 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:07.455672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07019b76-3bb2-4200-9f16-e4a64f1e10a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:07.457276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07019b76-3bb2-4200-9f16-e4a64f1e10a9 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:07.600025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 540/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:17:07 2022] POST /identity/v3/users => generated 286 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:17:09.793540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdfa9b-319b-4934-9fd4-90dbd1081c87 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:09.794159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdfa9b-319b-4934-9fd4-90dbd1081c87 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:09.794159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdfa9b-319b-4934-9fd4-90dbd1081c87 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:09.795497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 571/1111] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:09.801479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350da8a-2c3b-4255-b981-a4255f6c1607 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:09.802040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350da8a-2c3b-4255-b981-a4255f6c1607 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:09.802040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350da8a-2c3b-4255-b981-a4255f6c1607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:09.821593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-c350da8a-2c3b-4255-b981-a4255f6c1607 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:09.822861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c350da8a-2c3b-4255-b981-a4255f6c1607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:09.859183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:09.903048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 541/1112] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:09.928155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdfa9b-319b-4934-9fd4-90dbd1081c87 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:10.041351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdfa9b-319b-4934-9fd4-90dbd1081c87 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:10.070634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:10.079659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f675409-f2d0-43e8-bfce-8f1110e00954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:10.085188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f675409-f2d0-43e8-bfce-8f1110e00954 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:10.085188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f675409-f2d0-43e8-bfce-8f1110e00954 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:10.085188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f675409-f2d0-43e8-bfce-8f1110e00954 None admin] PATH_INFO: `/v3/roles/296546e6917a4eb6b58cac525a76f8a6` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:10.085188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f675409-f2d0-43e8-bfce-8f1110e00954 None admin] RBAC: Authorizing `identity:get_role(role_id=296546e6917a4eb6b58cac525a76f8a6)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:10.085188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f675409-f2d0-43e8-bfce-8f1110e00954 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:10.085659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 572/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:09 2022] GET /identity/v3/roles/296546e6917a4eb6b58cac525a76f8a6 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:10.091798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350da8a-2c3b-4255-b981-a4255f6c1607 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:10.211357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350da8a-2c3b-4255-b981-a4255f6c1607 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:10.246764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea1052-325d-4ab0-896d-2ac2de10b709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:10.247677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1052-325d-4ab0-896d-2ac2de10b709 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:10.247958 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1052-325d-4ab0-896d-2ac2de10b709 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:10.248231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea1052-325d-4ab0-896d-2ac2de10b709 None admin] PATH_INFO: `/v3/users/12632ce7b2fe4b97ba4762b61f8a5034` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:10.263781 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea1052-325d-4ab0-896d-2ac2de10b709 None admin] RBAC: Authorizing `identity:get_user(user_id=12632ce7b2fe4b97ba4762b61f8a5034)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:10.267683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea1052-325d-4ab0-896d-2ac2de10b709 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:10.267683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 542/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:10 2022] GET /identity/v3/users/12632ce7b2fe4b97ba4762b61f8a5034 => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:10.273190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f675409-f2d0-43e8-bfce-8f1110e00954 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:10.380350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f675409-f2d0-43e8-bfce-8f1110e00954 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:10.419790 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82796da-db95-4a62-9f25-55f297b48232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:10.422820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82796da-db95-4a62-9f25-55f297b48232 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:10.422820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82796da-db95-4a62-9f25-55f297b48232 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:10.422820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82796da-db95-4a62-9f25-55f297b48232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:10.488242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:10.495992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82796da-db95-4a62-9f25-55f297b48232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:10.497564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82796da-db95-4a62-9f25-55f297b48232 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:10.604154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 573/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:17:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:17:10.613879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea1052-325d-4ab0-896d-2ac2de10b709 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:10.655604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:10.768254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea1052-325d-4ab0-896d-2ac2de10b709 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:10.807087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f8a8d-17d0-4a5d-aa15-d7c153989b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:10.807994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f8a8d-17d0-4a5d-aa15-d7c153989b92 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:10.808274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f8a8d-17d0-4a5d-aa15-d7c153989b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:10.808547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f8a8d-17d0-4a5d-aa15-d7c153989b92 None admin] PATH_INFO: `/v3/projects/f51d580536ff42a3a157af7405926881` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:10.809773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f8a8d-17d0-4a5d-aa15-d7c153989b92 None admin] RBAC: Authorizing `identity:get_project(project_id=f51d580536ff42a3a157af7405926881)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:10.811211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f8a8d-17d0-4a5d-aa15-d7c153989b92 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:10.813014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 543/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:10 2022] GET /identity/v3/projects/f51d580536ff42a3a157af7405926881 => generated 311 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:10.819345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82796da-db95-4a62-9f25-55f297b48232 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:10.921148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:10.927655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82796da-db95-4a62-9f25-55f297b48232 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:10.962890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6b4b8-8436-48d9-91ae-5e243732c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:10.963573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6b4b8-8436-48d9-91ae-5e243732c9ca None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:10.963860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6b4b8-8436-48d9-91ae-5e243732c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:10.964142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6b4b8-8436-48d9-91ae-5e243732c9ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:10.965003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6b4b8-8436-48d9-91ae-5e243732c9ca None admin] RBAC: Adding query filter params (role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=f51d580536ff42a3a157af7405926881, user.id=12632ce7b2fe4b97ba4762b61f8a5034) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:10.965419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6b4b8-8436-48d9-91ae-5e243732c9ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:10.966982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6b4b8-8436-48d9-91ae-5e243732c9ca None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:10.973036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 574/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:17:10 2022] GET /identity/v3/role_assignments?user.id=12632ce7b2fe4b97ba4762b61f8a5034&scope.project.id=f51d580536ff42a3a157af7405926881&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:13.077643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16aae0-f953-42a1-9c69-25e2cec02364 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:13.078788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16aae0-f953-42a1-9c69-25e2cec02364 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:13.078788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16aae0-f953-42a1-9c69-25e2cec02364 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:13.080095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 544/1118] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:13.085421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d1281-99ac-4ad6-8495-082b795ffce0 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:13.085804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d1281-99ac-4ad6-8495-082b795ffce0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:13.085988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d1281-99ac-4ad6-8495-082b795ffce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:13.103162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-1e3d1281-99ac-4ad6-8495-082b795ffce0 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:13.104837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1e3d1281-99ac-4ad6-8495-082b795ffce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:13.174780 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 575/1119] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:13.198226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16aae0-f953-42a1-9c69-25e2cec02364 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:13.299645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:13.307687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16aae0-f953-42a1-9c69-25e2cec02364 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:13.346023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f05c1d-4731-499c-b267-53537e813cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:13.346879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f05c1d-4731-499c-b267-53537e813cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:13.347178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f05c1d-4731-499c-b267-53537e813cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:13.347461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f05c1d-4731-499c-b267-53537e813cb2 None admin] PATH_INFO: `/v3/roles/296546e6917a4eb6b58cac525a76f8a6` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:13.349198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f05c1d-4731-499c-b267-53537e813cb2 None admin] RBAC: Authorizing `identity:get_role(role_id=296546e6917a4eb6b58cac525a76f8a6)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:13.350691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f05c1d-4731-499c-b267-53537e813cb2 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:13.352202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 545/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:13 2022] GET /identity/v3/roles/296546e6917a4eb6b58cac525a76f8a6 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:13.362588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d1281-99ac-4ad6-8495-082b795ffce0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:13.453986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:13.466221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d1281-99ac-4ad6-8495-082b795ffce0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:13.501949 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd19dd-8bb0-475f-b80b-e0e2084aeff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:13.502833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd19dd-8bb0-475f-b80b-e0e2084aeff3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:13.503112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd19dd-8bb0-475f-b80b-e0e2084aeff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:13.503387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd19dd-8bb0-475f-b80b-e0e2084aeff3 None admin] PATH_INFO: `/v3/users/12632ce7b2fe4b97ba4762b61f8a5034` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:13.504626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd19dd-8bb0-475f-b80b-e0e2084aeff3 None admin] RBAC: Authorizing `identity:get_user(user_id=12632ce7b2fe4b97ba4762b61f8a5034)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:13.506077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd19dd-8bb0-475f-b80b-e0e2084aeff3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:13.507946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 576/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:13 2022] GET /identity/v3/users/12632ce7b2fe4b97ba4762b61f8a5034 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:13.514247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f05c1d-4731-499c-b267-53537e813cb2 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:13.618325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f05c1d-4731-499c-b267-53537e813cb2 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:13.658725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e533f-96b3-4749-8d87-ced80d30f0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:13.659621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e533f-96b3-4749-8d87-ced80d30f0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:13.659906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e533f-96b3-4749-8d87-ced80d30f0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:13.660182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e533f-96b3-4749-8d87-ced80d30f0c2 None admin] PATH_INFO: `/v3/projects/f51d580536ff42a3a157af7405926881` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:13.661427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e533f-96b3-4749-8d87-ced80d30f0c2 None admin] RBAC: Authorizing `identity:get_project(project_id=f51d580536ff42a3a157af7405926881)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:13.662881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e533f-96b3-4749-8d87-ced80d30f0c2 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:13.664677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 546/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:13 2022] GET /identity/v3/projects/f51d580536ff42a3a157af7405926881 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:13.671179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd19dd-8bb0-475f-b80b-e0e2084aeff3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:13.782789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd19dd-8bb0-475f-b80b-e0e2084aeff3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:13.821660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1cef3-7031-41a8-a190-7bb64a9caff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:13.822562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1cef3-7031-41a8-a190-7bb64a9caff8 None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:13.822843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1cef3-7031-41a8-a190-7bb64a9caff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:13.823117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1cef3-7031-41a8-a190-7bb64a9caff8 None admin] PATH_INFO: `/v3/projects/f51d580536ff42a3a157af7405926881/users/12632ce7b2fe4b97ba4762b61f8a5034/roles/296546e6917a4eb6b58cac525a76f8a6` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:13.825456 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1cef3-7031-41a8-a190-7bb64a9caff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f51d580536ff42a3a157af7405926881, user_id=12632ce7b2fe4b97ba4762b61f8a5034, role_id=296546e6917a4eb6b58cac525a76f8a6)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:13.826919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1cef3-7031-41a8-a190-7bb64a9caff8 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:13.894274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 577/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:17:13 2022] PUT /identity/v3/projects/f51d580536ff42a3a157af7405926881/users/12632ce7b2fe4b97ba4762b61f8a5034/roles/296546e6917a4eb6b58cac525a76f8a6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:17:16.003829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480179-5845-4fbb-ab4f-c011267fdc9a None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:16.004491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480179-5845-4fbb-ab4f-c011267fdc9a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:16.004491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480179-5845-4fbb-ab4f-c011267fdc9a None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:16.005958 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 547/1124] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:16.012288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91444-a772-4f2c-9ab2-4b8d9ffeb72a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:16.012844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91444-a772-4f2c-9ab2-4b8d9ffeb72a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:16.012844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91444-a772-4f2c-9ab2-4b8d9ffeb72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:16.032583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2ad91444-a772-4f2c-9ab2-4b8d9ffeb72a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:16.033887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2ad91444-a772-4f2c-9ab2-4b8d9ffeb72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:16.046199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:16.104682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 578/1125] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:17:16.128574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480179-5845-4fbb-ab4f-c011267fdc9a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:16.160839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:16.240232 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480179-5845-4fbb-ab4f-c011267fdc9a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:16.280193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf7a61-fed0-4e24-a7d8-7a20f5baddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:16.281119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf7a61-fed0-4e24-a7d8-7a20f5baddc9 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:16.281913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf7a61-fed0-4e24-a7d8-7a20f5baddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:16.282182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf7a61-fed0-4e24-a7d8-7a20f5baddc9 None admin] PATH_INFO: `/v3/roles/296546e6917a4eb6b58cac525a76f8a6` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:16.283766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf7a61-fed0-4e24-a7d8-7a20f5baddc9 None admin] RBAC: Authorizing `identity:get_role(role_id=296546e6917a4eb6b58cac525a76f8a6)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:16.285191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf7a61-fed0-4e24-a7d8-7a20f5baddc9 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:16.286647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 548/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:16 2022] GET /identity/v3/roles/296546e6917a4eb6b58cac525a76f8a6 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:16.293091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad91444-a772-4f2c-9ab2-4b8d9ffeb72a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:16.404327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad91444-a772-4f2c-9ab2-4b8d9ffeb72a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:16.444434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c25acd-0a1a-4b74-9b98-18dc413d9bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:16.445397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c25acd-0a1a-4b74-9b98-18dc413d9bcb None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:16.445729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c25acd-0a1a-4b74-9b98-18dc413d9bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:16.446057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c25acd-0a1a-4b74-9b98-18dc413d9bcb None admin] PATH_INFO: `/v3/users/12632ce7b2fe4b97ba4762b61f8a5034` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:16.447404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c25acd-0a1a-4b74-9b98-18dc413d9bcb None admin] RBAC: Authorizing `identity:get_user(user_id=12632ce7b2fe4b97ba4762b61f8a5034)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:16.448979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c25acd-0a1a-4b74-9b98-18dc413d9bcb None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:16.450752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 579/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:16 2022] GET /identity/v3/users/12632ce7b2fe4b97ba4762b61f8a5034 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:16.457722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf7a61-fed0-4e24-a7d8-7a20f5baddc9 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:16.571030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:16.575037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf7a61-fed0-4e24-a7d8-7a20f5baddc9 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:16.613736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2fe4b-75d8-4196-b15e-d5a7b7f1c05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:16.614397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2fe4b-75d8-4196-b15e-d5a7b7f1c05c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:16.614699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2fe4b-75d8-4196-b15e-d5a7b7f1c05c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:16.614977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2fe4b-75d8-4196-b15e-d5a7b7f1c05c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:16.688907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2fe4b-75d8-4196-b15e-d5a7b7f1c05c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:16.690379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2fe4b-75d8-4196-b15e-d5a7b7f1c05c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:16.795062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:16.801264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 549/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:17:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:17:16.807642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c25acd-0a1a-4b74-9b98-18dc413d9bcb None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:16.829315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:16.952782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c25acd-0a1a-4b74-9b98-18dc413d9bcb None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:16.989458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff9410-ff7a-4430-948a-7a94df27a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:16.990328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff9410-ff7a-4430-948a-7a94df27a5bf None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:16.990609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff9410-ff7a-4430-948a-7a94df27a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:16.990889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff9410-ff7a-4430-948a-7a94df27a5bf None admin] PATH_INFO: `/v3/projects/f51d580536ff42a3a157af7405926881` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:16.992169 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff9410-ff7a-4430-948a-7a94df27a5bf None admin] RBAC: Authorizing `identity:get_project(project_id=f51d580536ff42a3a157af7405926881)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:16.993621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff9410-ff7a-4430-948a-7a94df27a5bf None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:16.995366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 580/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:16 2022] GET /identity/v3/projects/f51d580536ff42a3a157af7405926881 => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:17.004333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2fe4b-75d8-4196-b15e-d5a7b7f1c05c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:17.108890 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2fe4b-75d8-4196-b15e-d5a7b7f1c05c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:17.145424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01048a-2e74-4168-88e5-e30a600abcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:17.146165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01048a-2e74-4168-88e5-e30a600abcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:17.146473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01048a-2e74-4168-88e5-e30a600abcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:17.146773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01048a-2e74-4168-88e5-e30a600abcc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:17.147693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01048a-2e74-4168-88e5-e30a600abcc0 None admin] RBAC: Adding query filter params (role.id=296546e6917a4eb6b58cac525a76f8a6, scope.project.id=f51d580536ff42a3a157af7405926881, user.id=12632ce7b2fe4b97ba4762b61f8a5034) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:17.148035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01048a-2e74-4168-88e5-e30a600abcc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:17.149601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01048a-2e74-4168-88e5-e30a600abcc0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:17.157800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 550/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:17:17 2022] GET /identity/v3/role_assignments?user.id=12632ce7b2fe4b97ba4762b61f8a5034&scope.project.id=f51d580536ff42a3a157af7405926881&role.id=296546e6917a4eb6b58cac525a76f8a6 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:19.275142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9fd3-ddee-4b42-bb4d-5d6ef80cd7d5 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:19.275785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9fd3-ddee-4b42-bb4d-5d6ef80cd7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:19.275785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9fd3-ddee-4b42-bb4d-5d6ef80cd7d5 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:19.277452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 581/1131] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:17:19.283621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614bbd1-290b-45e5-bf62-a9792ae0f8e9 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:19.284113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614bbd1-290b-45e5-bf62-a9792ae0f8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:19.284509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614bbd1-290b-45e5-bf62-a9792ae0f8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:19.301887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f614bbd1-290b-45e5-bf62-a9792ae0f8e9 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:19.303206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f614bbd1-290b-45e5-bf62-a9792ae0f8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:19.376962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:19.387252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 551/1132] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:19.413099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc9fd3-ddee-4b42-bb4d-5d6ef80cd7d5 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:19.495802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:19.515112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc9fd3-ddee-4b42-bb4d-5d6ef80cd7d5 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:19.556388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4fdc0-c094-45e0-81b6-a77d4f75a889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:19.557106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4fdc0-c094-45e0-81b6-a77d4f75a889 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:19.557385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4fdc0-c094-45e0-81b6-a77d4f75a889 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:19.557656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4fdc0-c094-45e0-81b6-a77d4f75a889 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:19.558912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4fdc0-c094-45e0-81b6-a77d4f75a889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:19.560383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4fdc0-c094-45e0-81b6-a77d4f75a889 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:19.562034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 582/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:17:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:19.568724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614bbd1-290b-45e5-bf62-a9792ae0f8e9 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:19.678724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614bbd1-290b-45e5-bf62-a9792ae0f8e9 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:19.717992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da4a59-3d49-4e88-a433-5accad7121cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:19.718696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da4a59-3d49-4e88-a433-5accad7121cf None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:19.718981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da4a59-3d49-4e88-a433-5accad7121cf None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:19.719262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da4a59-3d49-4e88-a433-5accad7121cf None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:19.720027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da4a59-3d49-4e88-a433-5accad7121cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:19.721479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da4a59-3d49-4e88-a433-5accad7121cf None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:19.822116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 552/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:17:19 2022] POST /identity/v3/projects => generated 311 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:17:21.936883 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d3632-f373-4bd1-9f62-036e8e9741a0 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:21.937923 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d3632-f373-4bd1-9f62-036e8e9741a0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:21.938016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d3632-f373-4bd1-9f62-036e8e9741a0 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:21.939343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 583/1135] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:21.944945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d21c6-c45b-4389-a79e-96b6ac6ea35b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:21.945431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d21c6-c45b-4389-a79e-96b6ac6ea35b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:21.945579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d21c6-c45b-4389-a79e-96b6ac6ea35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:21.963390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-616d21c6-c45b-4389-a79e-96b6ac6ea35b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:21.964638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-616d21c6-c45b-4389-a79e-96b6ac6ea35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:22.007353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:22.034946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 553/1136] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:22.060406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d3632-f373-4bd1-9f62-036e8e9741a0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:22.175697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d3632-f373-4bd1-9f62-036e8e9741a0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:22.213392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a09016-5215-4036-b4f7-44d9d451858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:22.214110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a09016-5215-4036-b4f7-44d9d451858e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:22.214393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a09016-5215-4036-b4f7-44d9d451858e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:22.214672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a09016-5215-4036-b4f7-44d9d451858e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:22.215957 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a09016-5215-4036-b4f7-44d9d451858e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:22.217417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a09016-5215-4036-b4f7-44d9d451858e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:22.219150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 584/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:17:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:22.225284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d21c6-c45b-4389-a79e-96b6ac6ea35b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:22.349785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d21c6-c45b-4389-a79e-96b6ac6ea35b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:22.385831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675ef62-ca6f-4eb9-9034-4747a88dbc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:22.386496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675ef62-ca6f-4eb9-9034-4747a88dbc31 None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:22.386882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675ef62-ca6f-4eb9-9034-4747a88dbc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:22.387165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675ef62-ca6f-4eb9-9034-4747a88dbc31 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:22.387920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675ef62-ca6f-4eb9-9034-4747a88dbc31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:22.389364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675ef62-ca6f-4eb9-9034-4747a88dbc31 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:22.534937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 554/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:17:22 2022] POST /identity/v3/users => generated 286 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:17:24.693219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5909108-98cc-4269-980f-23ca0b70e6c6 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:24.693729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5909108-98cc-4269-980f-23ca0b70e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:24.693861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5909108-98cc-4269-980f-23ca0b70e6c6 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:24.695171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 585/1139] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:24.700899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d39db-90a5-4a3b-8f7f-d66fb203161f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:24.701374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d39db-90a5-4a3b-8f7f-d66fb203161f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:24.701486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d39db-90a5-4a3b-8f7f-d66fb203161f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:24.718803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-700d39db-90a5-4a3b-8f7f-d66fb203161f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:24.720051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-700d39db-90a5-4a3b-8f7f-d66fb203161f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:24.745239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:24.790211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 555/1140] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:24.814826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5909108-98cc-4269-980f-23ca0b70e6c6 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:24.834252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:24.925397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5909108-98cc-4269-980f-23ca0b70e6c6 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:24.961161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd01cce-bbc6-407e-b530-68482f49c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:24.961833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01cce-bbc6-407e-b530-68482f49c44d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:24.962117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01cce-bbc6-407e-b530-68482f49c44d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:24.962393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01cce-bbc6-407e-b530-68482f49c44d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:24.974005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd01cce-bbc6-407e-b530-68482f49c44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-2dd01cce-bbc6-407e-b530-68482f49c44d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:17:24.982008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:24.983228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:24.984943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 586/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:17:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:24.990793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d39db-90a5-4a3b-8f7f-d66fb203161f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:25.160347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d39db-90a5-4a3b-8f7f-d66fb203161f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:25.221195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36d4fd-81de-4185-8338-83c41c719a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:25.222125 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36d4fd-81de-4185-8338-83c41c719a0c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:25.222677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36d4fd-81de-4185-8338-83c41c719a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:25.223190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36d4fd-81de-4185-8338-83c41c719a0c None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:25.224744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36d4fd-81de-4185-8338-83c41c719a0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:25.225123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36d4fd-81de-4185-8338-83c41c719a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:25.227743 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36d4fd-81de-4185-8338-83c41c719a0c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:25.243401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 556/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:17:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:25.251866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd01cce-bbc6-407e-b530-68482f49c44d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:25.340428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:25.362450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd01cce-bbc6-407e-b530-68482f49c44d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:25.397144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2edda-b0d4-47f0-9900-781844eeea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:25.398039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2edda-b0d4-47f0-9900-781844eeea22 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:25.398322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2edda-b0d4-47f0-9900-781844eeea22 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:25.398598 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2edda-b0d4-47f0-9900-781844eeea22 None admin] PATH_INFO: `/v3/users/aadd0c91b0664b51a45a96fa4af12c84` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:25.416373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2edda-b0d4-47f0-9900-781844eeea22 None admin] RBAC: Authorizing `identity:get_user(user_id=aadd0c91b0664b51a45a96fa4af12c84)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:25.417849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2edda-b0d4-47f0-9900-781844eeea22 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:25.419689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 587/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:25 2022] GET /identity/v3/users/aadd0c91b0664b51a45a96fa4af12c84 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:25.426590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36d4fd-81de-4185-8338-83c41c719a0c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:25.452364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:25.555485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36d4fd-81de-4185-8338-83c41c719a0c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:25.595055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce5969-a074-4a92-bd68-3f41e2e3567d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:25.595747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce5969-a074-4a92-bd68-3f41e2e3567d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:25.596033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce5969-a074-4a92-bd68-3f41e2e3567d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:25.596311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce5969-a074-4a92-bd68-3f41e2e3567d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:25.675854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce5969-a074-4a92-bd68-3f41e2e3567d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:25.677333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce5969-a074-4a92-bd68-3f41e2e3567d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:25.704372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:25.791680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 557/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:17:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:17:25.798504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2edda-b0d4-47f0-9900-781844eeea22 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:25.912245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2edda-b0d4-47f0-9900-781844eeea22 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:25.950163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0441d-868e-4958-b64d-f8346d4ad819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:25.951041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0441d-868e-4958-b64d-f8346d4ad819 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:25.951324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0441d-868e-4958-b64d-f8346d4ad819 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:25.951628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0441d-868e-4958-b64d-f8346d4ad819 None admin] PATH_INFO: `/v3/projects/d9d79c46fe514129903c56bca8d87829` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:25.953367 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0441d-868e-4958-b64d-f8346d4ad819 None admin] RBAC: Authorizing `identity:get_project(project_id=d9d79c46fe514129903c56bca8d87829)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:25.954806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0441d-868e-4958-b64d-f8346d4ad819 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:25.956591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 588/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:25 2022] GET /identity/v3/projects/d9d79c46fe514129903c56bca8d87829 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:25.963725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce5969-a074-4a92-bd68-3f41e2e3567d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:26.081583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce5969-a074-4a92-bd68-3f41e2e3567d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:26.108867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:26.120560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0a532-3809-48a2-8ce2-f0e3c65e0cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:26.122071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0a532-3809-48a2-8ce2-f0e3c65e0cee None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:26.122071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0a532-3809-48a2-8ce2-f0e3c65e0cee None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:26.122071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0a532-3809-48a2-8ce2-f0e3c65e0cee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:26.122648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0a532-3809-48a2-8ce2-f0e3c65e0cee None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=d9d79c46fe514129903c56bca8d87829, user.id=aadd0c91b0664b51a45a96fa4af12c84) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:26.122955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0a532-3809-48a2-8ce2-f0e3c65e0cee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:26.124424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0a532-3809-48a2-8ce2-f0e3c65e0cee None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:26.130313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 558/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:17:25 2022] GET /identity/v3/role_assignments?user.id=aadd0c91b0664b51a45a96fa4af12c84&scope.project.id=d9d79c46fe514129903c56bca8d87829&role.id=03b724e03f704f2ab338b64025a3db6d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:28.219670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae3295-a022-42c3-915b-123aa1a999da None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:28.220222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae3295-a022-42c3-915b-123aa1a999da None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:28.220222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae3295-a022-42c3-915b-123aa1a999da None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:28.221583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 589/1147] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:28.227331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80689c2d-09ec-452e-a5e6-3718e2bc960b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:28.227807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80689c2d-09ec-452e-a5e6-3718e2bc960b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:28.227921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80689c2d-09ec-452e-a5e6-3718e2bc960b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:28.244786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-80689c2d-09ec-452e-a5e6-3718e2bc960b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:28.245967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-80689c2d-09ec-452e-a5e6-3718e2bc960b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:28.323159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 559/1148] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:28.347117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae3295-a022-42c3-915b-123aa1a999da None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:28.351082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:28.454357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae3295-a022-42c3-915b-123aa1a999da None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:28.488895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cfc23-d3ea-434f-80e1-95acf3d8628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:28.489698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cfc23-d3ea-434f-80e1-95acf3d8628c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:28.489980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cfc23-d3ea-434f-80e1-95acf3d8628c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:28.490259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cfc23-d3ea-434f-80e1-95acf3d8628c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:28.501129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cfc23-d3ea-434f-80e1-95acf3d8628c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-8f6cfc23-d3ea-434f-80e1-95acf3d8628c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:17:28.508972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:28.510082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:28.511050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 590/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:17:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:28.517036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80689c2d-09ec-452e-a5e6-3718e2bc960b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:28.616147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80689c2d-09ec-452e-a5e6-3718e2bc960b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:28.633115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:28.651889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e9db8-13d6-49e4-a629-11a9e9fc364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:28.658107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e9db8-13d6-49e4-a629-11a9e9fc364d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:28.658107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e9db8-13d6-49e4-a629-11a9e9fc364d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:28.658107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e9db8-13d6-49e4-a629-11a9e9fc364d None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:28.658107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e9db8-13d6-49e4-a629-11a9e9fc364d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:28.658107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e9db8-13d6-49e4-a629-11a9e9fc364d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:28.658107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e9db8-13d6-49e4-a629-11a9e9fc364d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:28.664775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 560/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:17:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:28.670620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cfc23-d3ea-434f-80e1-95acf3d8628c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:28.736053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:28.775412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cfc23-d3ea-434f-80e1-95acf3d8628c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:28.810695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2415-07b9-4d27-898f-fb96e1529725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:28.811597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2415-07b9-4d27-898f-fb96e1529725 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:28.812589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2415-07b9-4d27-898f-fb96e1529725 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:28.812867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2415-07b9-4d27-898f-fb96e1529725 None admin] PATH_INFO: `/v3/users/aadd0c91b0664b51a45a96fa4af12c84` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:28.814089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2415-07b9-4d27-898f-fb96e1529725 None admin] RBAC: Authorizing `identity:get_user(user_id=aadd0c91b0664b51a45a96fa4af12c84)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:28.815564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2415-07b9-4d27-898f-fb96e1529725 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:28.817217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 591/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:28 2022] GET /identity/v3/users/aadd0c91b0664b51a45a96fa4af12c84 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:28.823072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e9db8-13d6-49e4-a629-11a9e9fc364d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:28.939664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e9db8-13d6-49e4-a629-11a9e9fc364d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:28.973788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f1cee-06c2-4ddc-9c10-a325ffdf2e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:28.974659 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f1cee-06c2-4ddc-9c10-a325ffdf2e87 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:28.974942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f1cee-06c2-4ddc-9c10-a325ffdf2e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:28.975220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f1cee-06c2-4ddc-9c10-a325ffdf2e87 None admin] PATH_INFO: `/v3/projects/d9d79c46fe514129903c56bca8d87829` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:28.976451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f1cee-06c2-4ddc-9c10-a325ffdf2e87 None admin] RBAC: Authorizing `identity:get_project(project_id=d9d79c46fe514129903c56bca8d87829)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:28.977927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f1cee-06c2-4ddc-9c10-a325ffdf2e87 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:28.979706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 561/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:28 2022] GET /identity/v3/projects/d9d79c46fe514129903c56bca8d87829 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:28.986329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2415-07b9-4d27-898f-fb96e1529725 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:29.088333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2415-07b9-4d27-898f-fb96e1529725 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:29.116657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:29.121516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade3c5b-6bab-41a7-8232-583ce3a59fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:29.122433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade3c5b-6bab-41a7-8232-583ce3a59fd8 None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:29.122711 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade3c5b-6bab-41a7-8232-583ce3a59fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:29.122986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade3c5b-6bab-41a7-8232-583ce3a59fd8 None admin] PATH_INFO: `/v3/projects/d9d79c46fe514129903c56bca8d87829/users/aadd0c91b0664b51a45a96fa4af12c84/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:29.125714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade3c5b-6bab-41a7-8232-583ce3a59fd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9d79c46fe514129903c56bca8d87829, user_id=aadd0c91b0664b51a45a96fa4af12c84, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:29.127153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade3c5b-6bab-41a7-8232-583ce3a59fd8 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:29.271956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 592/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:17:28 2022] PUT /identity/v3/projects/d9d79c46fe514129903c56bca8d87829/users/aadd0c91b0664b51a45a96fa4af12c84/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:17:31.305620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bafee4-a905-466f-8b18-baf6e802bad1 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:31.306223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bafee4-a905-466f-8b18-baf6e802bad1 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:31.306223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bafee4-a905-466f-8b18-baf6e802bad1 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:31.307579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 562/1154] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:31.313984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfe576-8447-4283-85d0-2e40a3a4fc41 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:31.314528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfe576-8447-4283-85d0-2e40a3a4fc41 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:31.314528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfe576-8447-4283-85d0-2e40a3a4fc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:31.339701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5bbfe576-8447-4283-85d0-2e40a3a4fc41 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:31.340937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5bbfe576-8447-4283-85d0-2e40a3a4fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:31.412926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 593/1155] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:17:31.437725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bafee4-a905-466f-8b18-baf6e802bad1 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:31.488646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:31.549096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bafee4-a905-466f-8b18-baf6e802bad1 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:31.588943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114e08-f82a-4a67-b48a-dc8289d76b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:31.589750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114e08-f82a-4a67-b48a-dc8289d76b92 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:31.590034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114e08-f82a-4a67-b48a-dc8289d76b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:31.590315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114e08-f82a-4a67-b48a-dc8289d76b92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:31.603140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa114e08-f82a-4a67-b48a-dc8289d76b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-aa114e08-f82a-4a67-b48a-dc8289d76b92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:17:31.611691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:31.612895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:17:31.613888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 563/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:17:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:31.623567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfe576-8447-4283-85d0-2e40a3a4fc41 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:31.725586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfe576-8447-4283-85d0-2e40a3a4fc41 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:31.749568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:31.760947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b990d11-c80b-4a2e-94d2-31a3132c84d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:31.761616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b990d11-c80b-4a2e-94d2-31a3132c84d2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:31.761894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b990d11-c80b-4a2e-94d2-31a3132c84d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:31.762168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b990d11-c80b-4a2e-94d2-31a3132c84d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:31.762913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b990d11-c80b-4a2e-94d2-31a3132c84d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:31.763216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b990d11-c80b-4a2e-94d2-31a3132c84d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:31.764615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b990d11-c80b-4a2e-94d2-31a3132c84d2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:31.773025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 594/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:17:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:31.779276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114e08-f82a-4a67-b48a-dc8289d76b92 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:31.884077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114e08-f82a-4a67-b48a-dc8289d76b92 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:31.891999 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:31.918790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56104d3-2ca2-4aea-b13d-4b3d5c95559d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:31.919730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56104d3-2ca2-4aea-b13d-4b3d5c95559d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:31.920016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56104d3-2ca2-4aea-b13d-4b3d5c95559d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:31.920382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56104d3-2ca2-4aea-b13d-4b3d5c95559d None admin] PATH_INFO: `/v3/users/aadd0c91b0664b51a45a96fa4af12c84` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:31.921748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56104d3-2ca2-4aea-b13d-4b3d5c95559d None admin] RBAC: Authorizing `identity:get_user(user_id=aadd0c91b0664b51a45a96fa4af12c84)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:31.923627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56104d3-2ca2-4aea-b13d-4b3d5c95559d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:31.925878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 564/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:31 2022] GET /identity/v3/users/aadd0c91b0664b51a45a96fa4af12c84 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:31.933854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b990d11-c80b-4a2e-94d2-31a3132c84d2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:32.041627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b990d11-c80b-4a2e-94d2-31a3132c84d2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:32.076912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80494e62-8e11-4f44-a502-0434cda86f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:32.078095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80494e62-8e11-4f44-a502-0434cda86f6c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:32.078378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80494e62-8e11-4f44-a502-0434cda86f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:32.078658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80494e62-8e11-4f44-a502-0434cda86f6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:32.134075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:32.151405 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80494e62-8e11-4f44-a502-0434cda86f6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:32.151874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80494e62-8e11-4f44-a502-0434cda86f6c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:32.259165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 595/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:17:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:17:32.266043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56104d3-2ca2-4aea-b13d-4b3d5c95559d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:32.395144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56104d3-2ca2-4aea-b13d-4b3d5c95559d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:32.430026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d193e-844c-4dc3-bdeb-664bc7b8b0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:32.430802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d193e-844c-4dc3-bdeb-664bc7b8b0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:32.431090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d193e-844c-4dc3-bdeb-664bc7b8b0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:32.431363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d193e-844c-4dc3-bdeb-664bc7b8b0d6 None admin] PATH_INFO: `/v3/projects/d9d79c46fe514129903c56bca8d87829` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:32.432629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d193e-844c-4dc3-bdeb-664bc7b8b0d6 None admin] RBAC: Authorizing `identity:get_project(project_id=d9d79c46fe514129903c56bca8d87829)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:32.434074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d193e-844c-4dc3-bdeb-664bc7b8b0d6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:32.435890 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 565/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:32 2022] GET /identity/v3/projects/d9d79c46fe514129903c56bca8d87829 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:32.442868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80494e62-8e11-4f44-a502-0434cda86f6c None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:32.534923 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:32.551592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80494e62-8e11-4f44-a502-0434cda86f6c None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:32.587272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f405b2-22c9-41ac-8adf-9506cfd13ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:32.588076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f405b2-22c9-41ac-8adf-9506cfd13ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:32.588363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f405b2-22c9-41ac-8adf-9506cfd13ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:32.588643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f405b2-22c9-41ac-8adf-9506cfd13ce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:32.589490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f405b2-22c9-41ac-8adf-9506cfd13ce3 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=d9d79c46fe514129903c56bca8d87829, user.id=aadd0c91b0664b51a45a96fa4af12c84) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:32.589798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f405b2-22c9-41ac-8adf-9506cfd13ce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:32.591243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f405b2-22c9-41ac-8adf-9506cfd13ce3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:32.597710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 596/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:17:32 2022] GET /identity/v3/role_assignments?user.id=aadd0c91b0664b51a45a96fa4af12c84&scope.project.id=d9d79c46fe514129903c56bca8d87829&role.id=03b724e03f704f2ab338b64025a3db6d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:34.708671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b838c6-6bdc-4c24-8fa7-30d499e29fe0 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:34.712102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b838c6-6bdc-4c24-8fa7-30d499e29fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:34.712102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b838c6-6bdc-4c24-8fa7-30d499e29fe0 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:34.712102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 566/1162] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:34.717083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c84cd-7cb8-440a-aca5-13606bd497be None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:34.717560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c84cd-7cb8-440a-aca5-13606bd497be None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:34.717666 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c84cd-7cb8-440a-aca5-13606bd497be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:34.736494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3a2c84cd-7cb8-440a-aca5-13606bd497be None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:34.737707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3a2c84cd-7cb8-440a-aca5-13606bd497be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:34.807499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 597/1163] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:34.830784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b838c6-6bdc-4c24-8fa7-30d499e29fe0 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:34.888808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:34.949970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b838c6-6bdc-4c24-8fa7-30d499e29fe0 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:34.988386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b8e2c-8b67-40a5-9540-24123742963c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:34.989150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b8e2c-8b67-40a5-9540-24123742963c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:34.989471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b8e2c-8b67-40a5-9540-24123742963c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:34.989753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b8e2c-8b67-40a5-9540-24123742963c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:35.067585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b8e2c-8b67-40a5-9540-24123742963c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:35.069089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b8e2c-8b67-40a5-9540-24123742963c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:35.117408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:35.176409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 567/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:17:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:17:35.183295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c84cd-7cb8-440a-aca5-13606bd497be None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:35.265641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:35.289220 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c84cd-7cb8-440a-aca5-13606bd497be None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:35.325309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25763e3-0200-4200-8f09-8862ec73cadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:35.326291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25763e3-0200-4200-8f09-8862ec73cadd None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:35.326573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25763e3-0200-4200-8f09-8862ec73cadd None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:35.326847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25763e3-0200-4200-8f09-8862ec73cadd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:35.335454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25763e3-0200-4200-8f09-8862ec73cadd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:35.336963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25763e3-0200-4200-8f09-8862ec73cadd None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-f25763e3-0200-4200-8f09-8862ec73cadd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:35.352657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:35.354709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:35.356447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 598/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:17:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:35.360182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b8e2c-8b67-40a5-9540-24123742963c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:35.482862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b8e2c-8b67-40a5-9540-24123742963c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:35.533662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61d397-c463-4130-9fed-9f9862fc061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:35.534247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61d397-c463-4130-9fed-9f9862fc061f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:35.534530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61d397-c463-4130-9fed-9f9862fc061f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:35.534806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61d397-c463-4130-9fed-9f9862fc061f None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:35.535598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61d397-c463-4130-9fed-9f9862fc061f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:35.535906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61d397-c463-4130-9fed-9f9862fc061f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:35.537288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61d397-c463-4130-9fed-9f9862fc061f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:35.547263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 568/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Mon Jun 27 09:17:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:37.679628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f7de0-15f8-4972-bdf3-c45d77924d85 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:37.680185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f7de0-15f8-4972-bdf3-c45d77924d85 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:37.680185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f7de0-15f8-4972-bdf3-c45d77924d85 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:37.681618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 599/1167] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:37.687814 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8aca5-4bc3-455d-b0a4-adf8635d7af9 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:37.688392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8aca5-4bc3-455d-b0a4-adf8635d7af9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:37.688392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8aca5-4bc3-455d-b0a4-adf8635d7af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:37.700723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:37.706724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-48c8aca5-4bc3-455d-b0a4-adf8635d7af9 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:37.708036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-48c8aca5-4bc3-455d-b0a4-adf8635d7af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:37.784586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 569/1168] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:17:37.809812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f7de0-15f8-4972-bdf3-c45d77924d85 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:37.927817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f7de0-15f8-4972-bdf3-c45d77924d85 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:37.964700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70fd6e-754b-4bd2-86bc-baa140ec40f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:37.965285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fd6e-754b-4bd2-86bc-baa140ec40f5 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:37.965565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fd6e-754b-4bd2-86bc-baa140ec40f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:37.965842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fd6e-754b-4bd2-86bc-baa140ec40f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:37.966603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70fd6e-754b-4bd2-86bc-baa140ec40f5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:37.968580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70fd6e-754b-4bd2-86bc-baa140ec40f5 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:38.088337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:38.096908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 600/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Mon Jun 27 09:17:37 2022] POST /identity/v3/domains => generated 267 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:17:40.290262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4cec-0d21-4ebc-9919-102f508ebc37 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:40.291153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4cec-0d21-4ebc-9919-102f508ebc37 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:40.291153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4cec-0d21-4ebc-9919-102f508ebc37 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:40.292298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 570/1170] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:17:40.298863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8f416-1284-424b-a4d5-fd4d95fa6688 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:40.299437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8f416-1284-424b-a4d5-fd4d95fa6688 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:40.299587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8f416-1284-424b-a4d5-fd4d95fa6688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:40.318868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-aed8f416-1284-424b-a4d5-fd4d95fa6688 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:40.320148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-aed8f416-1284-424b-a4d5-fd4d95fa6688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:40.402936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 601/1171] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:17:40.428675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d4cec-0d21-4ebc-9919-102f508ebc37 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:40.544081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d4cec-0d21-4ebc-9919-102f508ebc37 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:40.584956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96d3ef-1cfd-42f8-bce1-d411d31f43fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:40.585874 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96d3ef-1cfd-42f8-bce1-d411d31f43fc None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:40.586153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96d3ef-1cfd-42f8-bce1-d411d31f43fc None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:40.586428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96d3ef-1cfd-42f8-bce1-d411d31f43fc None admin] PATH_INFO: `/v3/domains/c3cb81aac9b64a3d8c1bbb4a2c2d53c4` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:40.589986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:40.600327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96d3ef-1cfd-42f8-bce1-d411d31f43fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=c3cb81aac9b64a3d8c1bbb4a2c2d53c4)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:40.607187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96d3ef-1cfd-42f8-bce1-d411d31f43fc None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:40.607187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 571/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Mon Jun 27 09:17:40 2022] GET /identity/v3/domains/c3cb81aac9b64a3d8c1bbb4a2c2d53c4 => generated 267 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:40.610295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8f416-1284-424b-a4d5-fd4d95fa6688 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:40.726071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8f416-1284-424b-a4d5-fd4d95fa6688 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:40.756327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:40.764219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded1eae-bc29-42ff-9f2f-bf0af55b6071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:40.768947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded1eae-bc29-42ff-9f2f-bf0af55b6071 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:40.768947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded1eae-bc29-42ff-9f2f-bf0af55b6071 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:40.768947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded1eae-bc29-42ff-9f2f-bf0af55b6071 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:40.768947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded1eae-bc29-42ff-9f2f-bf0af55b6071 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:40.768947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded1eae-bc29-42ff-9f2f-bf0af55b6071 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:40.981386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 602/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:17:40 2022] POST /identity/v3/projects => generated 361 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:17:43.129093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787407c-6a27-44cf-9076-3eb77addc86d None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:43.129933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787407c-6a27-44cf-9076-3eb77addc86d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:43.130119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787407c-6a27-44cf-9076-3eb77addc86d None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:43.131560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 572/1174] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:17:43.137505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc38d20-3181-4004-be1f-69817b31759e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:43.138044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc38d20-3181-4004-be1f-69817b31759e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:43.138044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc38d20-3181-4004-be1f-69817b31759e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:43.166535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8fc38d20-3181-4004-be1f-69817b31759e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:43.168916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8fc38d20-3181-4004-be1f-69817b31759e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:43.270990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 603/1175] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:43.308372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787407c-6a27-44cf-9076-3eb77addc86d None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:43.457460 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787407c-6a27-44cf-9076-3eb77addc86d None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:43.495471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f705837-3b96-4900-b539-90054c617f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:43.496380 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f705837-3b96-4900-b539-90054c617f8b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:43.496665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f705837-3b96-4900-b539-90054c617f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:43.497050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f705837-3b96-4900-b539-90054c617f8b None admin] PATH_INFO: `/v3/domains/c3cb81aac9b64a3d8c1bbb4a2c2d53c4` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:43.498307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f705837-3b96-4900-b539-90054c617f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=c3cb81aac9b64a3d8c1bbb4a2c2d53c4)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:43.500392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f705837-3b96-4900-b539-90054c617f8b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:43.502171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 573/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Mon Jun 27 09:17:43 2022] GET /identity/v3/domains/c3cb81aac9b64a3d8c1bbb4a2c2d53c4 => generated 267 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:43.508401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc38d20-3181-4004-be1f-69817b31759e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:43.618075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc38d20-3181-4004-be1f-69817b31759e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:43.630628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:43.655438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb9f67-cc76-4df6-85f2-01f3d5a9c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:43.656160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb9f67-cc76-4df6-85f2-01f3d5a9c1ed None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:43.656443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb9f67-cc76-4df6-85f2-01f3d5a9c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:43.656725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb9f67-cc76-4df6-85f2-01f3d5a9c1ed None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:43.657453 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb9f67-cc76-4df6-85f2-01f3d5a9c1ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:43.658951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb9f67-cc76-4df6-85f2-01f3d5a9c1ed None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:43.757605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 604/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:17:43 2022] POST /identity/v3/users => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:17:45.971830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d827eb-49f2-438a-8981-6835857df8e8 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:45.972392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d827eb-49f2-438a-8981-6835857df8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:45.972392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d827eb-49f2-438a-8981-6835857df8e8 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:45.973777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 574/1178] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:45.979326 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ccead-0669-4cd2-96d8-72b001fc8a02 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:45.979752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ccead-0669-4cd2-96d8-72b001fc8a02 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:45.979916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ccead-0669-4cd2-96d8-72b001fc8a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:45.997305 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a79ccead-0669-4cd2-96d8-72b001fc8a02 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:45.998564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a79ccead-0669-4cd2-96d8-72b001fc8a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:46.073348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 605/1179] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:46.098024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d827eb-49f2-438a-8981-6835857df8e8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:46.176386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:46.217773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d827eb-49f2-438a-8981-6835857df8e8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:46.255581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08867a-4ca5-433b-af6c-e011680a4eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:46.256398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08867a-4ca5-433b-af6c-e011680a4eed None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:46.256681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08867a-4ca5-433b-af6c-e011680a4eed None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:46.256987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08867a-4ca5-433b-af6c-e011680a4eed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:46.268164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08867a-4ca5-433b-af6c-e011680a4eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-9b08867a-4ca5-433b-af6c-e011680a4eed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:17:46.276558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:46.278023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:17:46.279704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 575/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:17:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:46.284820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ccead-0669-4cd2-96d8-72b001fc8a02 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:46.461800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:46.468137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ccead-0669-4cd2-96d8-72b001fc8a02 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:46.503463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab42989-7e0d-4af9-b050-a9f045ffb661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:46.504173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42989-7e0d-4af9-b050-a9f045ffb661 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:46.504457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42989-7e0d-4af9-b050-a9f045ffb661 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:46.504735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42989-7e0d-4af9-b050-a9f045ffb661 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:46.505492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab42989-7e0d-4af9-b050-a9f045ffb661 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:46.505857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab42989-7e0d-4af9-b050-a9f045ffb661 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:46.507219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab42989-7e0d-4af9-b050-a9f045ffb661 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:46.516120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 606/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:17:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:46.523479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08867a-4ca5-433b-af6c-e011680a4eed None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:46.635908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08867a-4ca5-433b-af6c-e011680a4eed None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:46.663570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:46.676132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deb6a1-d328-428a-902e-33857e1cf71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:46.677052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deb6a1-d328-428a-902e-33857e1cf71d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:46.677427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deb6a1-d328-428a-902e-33857e1cf71d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:46.677593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deb6a1-d328-428a-902e-33857e1cf71d None admin] PATH_INFO: `/v3/users/557013178239455f86b13cde919951a4` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:46.694280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1deb6a1-d328-428a-902e-33857e1cf71d None admin] RBAC: Authorizing `identity:get_user(user_id=557013178239455f86b13cde919951a4)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:46.695802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1deb6a1-d328-428a-902e-33857e1cf71d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:46.697714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 576/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:46 2022] GET /identity/v3/users/557013178239455f86b13cde919951a4 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:46.706177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab42989-7e0d-4af9-b050-a9f045ffb661 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:46.818959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab42989-7e0d-4af9-b050-a9f045ffb661 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:46.870131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a077f-d7c5-450c-b659-5e267401e75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:46.871223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a077f-d7c5-450c-b659-5e267401e75f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:46.871638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a077f-d7c5-450c-b659-5e267401e75f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:46.872008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a077f-d7c5-450c-b659-5e267401e75f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:46.902252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:46.946010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a077f-d7c5-450c-b659-5e267401e75f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:46.947976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a077f-d7c5-450c-b659-5e267401e75f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:47.056646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 607/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:17:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:17:47.066078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deb6a1-d328-428a-902e-33857e1cf71d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:47.174893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deb6a1-d328-428a-902e-33857e1cf71d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:47.215058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef85c46-ff4e-4bc8-a601-e855aa3ffb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:47.215957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef85c46-ff4e-4bc8-a601-e855aa3ffb3e None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:47.216237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef85c46-ff4e-4bc8-a601-e855aa3ffb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:47.216507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef85c46-ff4e-4bc8-a601-e855aa3ffb3e None admin] PATH_INFO: `/v3/projects/20236e52acc84b14a2d47b60cdd7ba5d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:47.217823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef85c46-ff4e-4bc8-a601-e855aa3ffb3e None admin] RBAC: Authorizing `identity:get_project(project_id=20236e52acc84b14a2d47b60cdd7ba5d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:47.219495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef85c46-ff4e-4bc8-a601-e855aa3ffb3e None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:47.221399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 577/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:47 2022] GET /identity/v3/projects/20236e52acc84b14a2d47b60cdd7ba5d => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:47.228055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a077f-d7c5-450c-b659-5e267401e75f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:47.297133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:47.344050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a077f-d7c5-450c-b659-5e267401e75f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:47.381606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09061a0-1579-429f-a82e-c673aa35ba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:47.382272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09061a0-1579-429f-a82e-c673aa35ba1a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:47.382552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09061a0-1579-429f-a82e-c673aa35ba1a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:47.382912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09061a0-1579-429f-a82e-c673aa35ba1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:47.383786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09061a0-1579-429f-a82e-c673aa35ba1a None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=20236e52acc84b14a2d47b60cdd7ba5d, user.id=557013178239455f86b13cde919951a4) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:47.384096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09061a0-1579-429f-a82e-c673aa35ba1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:47.385555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09061a0-1579-429f-a82e-c673aa35ba1a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:47.392344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 608/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:17:47 2022] GET /identity/v3/role_assignments?user.id=557013178239455f86b13cde919951a4&scope.project.id=20236e52acc84b14a2d47b60cdd7ba5d&role.id=03b724e03f704f2ab338b64025a3db6d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:17:49.481672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6500c44-c0d9-4c71-a447-743009fe4211 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:49.482278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6500c44-c0d9-4c71-a447-743009fe4211 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:49.482278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6500c44-c0d9-4c71-a447-743009fe4211 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:49.484327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 578/1186] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:17:49.490394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510984-9006-47c4-a19a-e44acce5f4d5 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:49.490968 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510984-9006-47c4-a19a-e44acce5f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:49.490968 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510984-9006-47c4-a19a-e44acce5f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:49.509246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4e510984-9006-47c4-a19a-e44acce5f4d5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:49.510560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4e510984-9006-47c4-a19a-e44acce5f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:49.584666 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 609/1187] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:49.624144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6500c44-c0d9-4c71-a447-743009fe4211 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:49.695801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:49.753759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6500c44-c0d9-4c71-a447-743009fe4211 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:49.789844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5497a-ca5e-47b4-b0dc-68459828200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:49.790689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5497a-ca5e-47b4-b0dc-68459828200b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:49.790997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5497a-ca5e-47b4-b0dc-68459828200b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:49.791269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5497a-ca5e-47b4-b0dc-68459828200b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:49.803389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5497a-ca5e-47b4-b0dc-68459828200b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-43d5497a-ca5e-47b4-b0dc-68459828200b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:17:49.811274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:49.812563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:17:49.813626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 579/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:49.820011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e510984-9006-47c4-a19a-e44acce5f4d5 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:49.880726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:49.931082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e510984-9006-47c4-a19a-e44acce5f4d5 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:49.969381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9fe1b-dd5c-4da1-b65c-2a47da0889a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:49.970064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9fe1b-dd5c-4da1-b65c-2a47da0889a2 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:49.970348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9fe1b-dd5c-4da1-b65c-2a47da0889a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:49.970631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9fe1b-dd5c-4da1-b65c-2a47da0889a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:49.971429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9fe1b-dd5c-4da1-b65c-2a47da0889a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:49.971774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9fe1b-dd5c-4da1-b65c-2a47da0889a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:49.973230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9fe1b-dd5c-4da1-b65c-2a47da0889a2 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:49.982182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 610/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:49.990175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5497a-ca5e-47b4-b0dc-68459828200b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:50.096066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5497a-ca5e-47b4-b0dc-68459828200b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:50.106725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:50.130984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7718dc4-41bb-4725-9bb8-b1018c4c45c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:50.131964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7718dc4-41bb-4725-9bb8-b1018c4c45c5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:50.132251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7718dc4-41bb-4725-9bb8-b1018c4c45c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:50.132526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7718dc4-41bb-4725-9bb8-b1018c4c45c5 None admin] PATH_INFO: `/v3/users/557013178239455f86b13cde919951a4` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:50.133829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7718dc4-41bb-4725-9bb8-b1018c4c45c5 None admin] RBAC: Authorizing `identity:get_user(user_id=557013178239455f86b13cde919951a4)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:50.135297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7718dc4-41bb-4725-9bb8-b1018c4c45c5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:50.137013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 580/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:49 2022] GET /identity/v3/users/557013178239455f86b13cde919951a4 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:50.143734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9fe1b-dd5c-4da1-b65c-2a47da0889a2 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:50.290244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9fe1b-dd5c-4da1-b65c-2a47da0889a2 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:50.304513 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:50.325361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae154e-9248-4cbf-bb72-778394a766df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:50.326238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae154e-9248-4cbf-bb72-778394a766df None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:50.326521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae154e-9248-4cbf-bb72-778394a766df None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:50.327363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae154e-9248-4cbf-bb72-778394a766df None admin] PATH_INFO: `/v3/projects/20236e52acc84b14a2d47b60cdd7ba5d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:50.328655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae154e-9248-4cbf-bb72-778394a766df None admin] RBAC: Authorizing `identity:get_project(project_id=20236e52acc84b14a2d47b60cdd7ba5d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:50.330119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae154e-9248-4cbf-bb72-778394a766df None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:50.332108 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 611/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:50 2022] GET /identity/v3/projects/20236e52acc84b14a2d47b60cdd7ba5d => generated 361 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:50.338352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7718dc4-41bb-4725-9bb8-b1018c4c45c5 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:50.450755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7718dc4-41bb-4725-9bb8-b1018c4c45c5 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:50.487971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1590eae-7e46-4764-9546-526cd0d04ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:50.488923 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1590eae-7e46-4764-9546-526cd0d04ca9 None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:50.489209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1590eae-7e46-4764-9546-526cd0d04ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:50.489488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1590eae-7e46-4764-9546-526cd0d04ca9 None admin] PATH_INFO: `/v3/projects/20236e52acc84b14a2d47b60cdd7ba5d/users/557013178239455f86b13cde919951a4/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:50.491575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1590eae-7e46-4764-9546-526cd0d04ca9 None admin] RBAC: Authorizing `identity:create_grant(project_id=20236e52acc84b14a2d47b60cdd7ba5d, user_id=557013178239455f86b13cde919951a4, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:50.493035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1590eae-7e46-4764-9546-526cd0d04ca9 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:50.585440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 581/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:17:50 2022] PUT /identity/v3/projects/20236e52acc84b14a2d47b60cdd7ba5d/users/557013178239455f86b13cde919951a4/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:17:52.656195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3297fe-205d-48fb-a248-69cc6fc0f54c None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:52.656697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3297fe-205d-48fb-a248-69cc6fc0f54c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:52.656697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3297fe-205d-48fb-a248-69cc6fc0f54c None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:52.658080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 612/1193] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:17:52.663490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cff31-7f51-4698-8f73-a2b39cac16f7 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:52.663977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cff31-7f51-4698-8f73-a2b39cac16f7 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:52.664224 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cff31-7f51-4698-8f73-a2b39cac16f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:52.681621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-eb0cff31-7f51-4698-8f73-a2b39cac16f7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:17:52.682815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-eb0cff31-7f51-4698-8f73-a2b39cac16f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:17:52.711841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:52.749901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 582/1194] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:17:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:17:52.774058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3297fe-205d-48fb-a248-69cc6fc0f54c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:52.878298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3297fe-205d-48fb-a248-69cc6fc0f54c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:52.915836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642fcad-b3c7-47a7-b5d5-267451989f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:52.916558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642fcad-b3c7-47a7-b5d5-267451989f92 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:52.916843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642fcad-b3c7-47a7-b5d5-267451989f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:52.917120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642fcad-b3c7-47a7-b5d5-267451989f92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:52.929842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642fcad-b3c7-47a7-b5d5-267451989f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-f642fcad-b3c7-47a7-b5d5-267451989f92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:17:52.937728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:17:52.938861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:17:52.939838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 613/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:17:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:17:52.945104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cff31-7f51-4698-8f73-a2b39cac16f7 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:53.042611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cff31-7f51-4698-8f73-a2b39cac16f7 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:53.076027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd817c5-f960-4bd8-b281-7fe55d17466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:53.077217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd817c5-f960-4bd8-b281-7fe55d17466d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:53.077498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd817c5-f960-4bd8-b281-7fe55d17466d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:53.077774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd817c5-f960-4bd8-b281-7fe55d17466d None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:53.078522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd817c5-f960-4bd8-b281-7fe55d17466d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:53.078841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd817c5-f960-4bd8-b281-7fe55d17466d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:53.080286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd817c5-f960-4bd8-b281-7fe55d17466d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:53.089032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 583/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:17:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:53.095478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642fcad-b3c7-47a7-b5d5-267451989f92 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:53.141477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:53.199578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642fcad-b3c7-47a7-b5d5-267451989f92 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:53.236647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c3312-cc06-4085-bde9-2a1aeea14060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:53.237538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c3312-cc06-4085-bde9-2a1aeea14060 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:53.237820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c3312-cc06-4085-bde9-2a1aeea14060 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:53.238098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c3312-cc06-4085-bde9-2a1aeea14060 None admin] PATH_INFO: `/v3/users/557013178239455f86b13cde919951a4` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:53.239345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c3312-cc06-4085-bde9-2a1aeea14060 None admin] RBAC: Authorizing `identity:get_user(user_id=557013178239455f86b13cde919951a4)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:53.240897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c3312-cc06-4085-bde9-2a1aeea14060 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:53.244390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 614/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Mon Jun 27 09:17:53 2022] GET /identity/v3/users/557013178239455f86b13cde919951a4 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:53.249009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd817c5-f960-4bd8-b281-7fe55d17466d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:53.283179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:53.353920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd817c5-f960-4bd8-b281-7fe55d17466d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:53.394033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf357ed-25d7-4a0d-8004-1eb0b7a8a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:53.394692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf357ed-25d7-4a0d-8004-1eb0b7a8a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:53.395099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf357ed-25d7-4a0d-8004-1eb0b7a8a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:53.395373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf357ed-25d7-4a0d-8004-1eb0b7a8a2a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:53.470221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf357ed-25d7-4a0d-8004-1eb0b7a8a2a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:53.471842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf357ed-25d7-4a0d-8004-1eb0b7a8a2a5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:53.505038 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:53.575868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 584/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:17:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:17:53.581875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c3312-cc06-4085-bde9-2a1aeea14060 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:53.686978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c3312-cc06-4085-bde9-2a1aeea14060 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:53.698619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:53.724225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b759a1c-d43c-4e2c-a56a-eb0cae83df8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:53.725016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b759a1c-d43c-4e2c-a56a-eb0cae83df8e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:53.725299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b759a1c-d43c-4e2c-a56a-eb0cae83df8e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:53.725619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b759a1c-d43c-4e2c-a56a-eb0cae83df8e None admin] PATH_INFO: `/v3/projects/20236e52acc84b14a2d47b60cdd7ba5d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:53.726916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b759a1c-d43c-4e2c-a56a-eb0cae83df8e None admin] RBAC: Authorizing `identity:get_project(project_id=20236e52acc84b14a2d47b60cdd7ba5d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:53.728426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b759a1c-d43c-4e2c-a56a-eb0cae83df8e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:53.730155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 615/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Mon Jun 27 09:17:53 2022] GET /identity/v3/projects/20236e52acc84b14a2d47b60cdd7ba5d => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:17:53.740727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf357ed-25d7-4a0d-8004-1eb0b7a8a2a5 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:17:53.866725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf357ed-25d7-4a0d-8004-1eb0b7a8a2a5 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:17:53.895316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:17:53.901093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c6c0d-45c8-45f0-a8c7-a858c32c22b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:17:53.902012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c6c0d-45c8-45f0-a8c7-a858c32c22b3 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:17:53.902357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c6c0d-45c8-45f0-a8c7-a858c32c22b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:17:53.902580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c6c0d-45c8-45f0-a8c7-a858c32c22b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:17:53.903413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c6c0d-45c8-45f0-a8c7-a858c32c22b3 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=20236e52acc84b14a2d47b60cdd7ba5d, user.id=557013178239455f86b13cde919951a4) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:17:53.903753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c6c0d-45c8-45f0-a8c7-a858c32c22b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:17:53.907670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c6c0d-45c8-45f0-a8c7-a858c32c22b3 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:17:53.911615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 585/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:17:53 2022] GET /identity/v3/role_assignments?user.id=557013178239455f86b13cde919951a4&scope.project.id=20236e52acc84b14a2d47b60cdd7ba5d&role.id=03b724e03f704f2ab338b64025a3db6d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:18:03.744434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=11 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:18:03.933238 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:13.446289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58bda4-c074-4fe7-b46d-38174b27a89c None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:13.447195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58bda4-c074-4fe7-b46d-38174b27a89c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:13.447195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58bda4-c074-4fe7-b46d-38174b27a89c None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:13.448287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 616/1201] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:13.454319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da5cad-5656-4ce8-aa0c-41711d3f1df5 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:13.457935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da5cad-5656-4ce8-aa0c-41711d3f1df5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:13.457935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da5cad-5656-4ce8-aa0c-41711d3f1df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:13.472204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-83da5cad-5656-4ce8-aa0c-41711d3f1df5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:13.473408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-83da5cad-5656-4ce8-aa0c-41711d3f1df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:13.541961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 586/1202] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:19:13.567740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58bda4-c074-4fe7-b46d-38174b27a89c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:13.675331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58bda4-c074-4fe7-b46d-38174b27a89c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:13.709693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cc44c-b1c0-4308-b7cf-dc96c1b3e4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:13.710512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cc44c-b1c0-4308-b7cf-dc96c1b3e4ca None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:13.710793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cc44c-b1c0-4308-b7cf-dc96c1b3e4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:13.711068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cc44c-b1c0-4308-b7cf-dc96c1b3e4ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:13.722564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cc44c-b1c0-4308-b7cf-dc96c1b3e4ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:13.724072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cc44c-b1c0-4308-b7cf-dc96c1b3e4ca None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-a24cc44c-b1c0-4308-b7cf-dc96c1b3e4ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:13.743334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:13.744616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:13.745805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:13.745805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:13.745805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:13.745805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:13.745805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 617/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:13.752067 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da5cad-5656-4ce8-aa0c-41711d3f1df5 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:13.861627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da5cad-5656-4ce8-aa0c-41711d3f1df5 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:13.894616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:13.900662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cca258-9b1e-4a99-9728-81a49eb16e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:13.901341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cca258-9b1e-4a99-9728-81a49eb16e97 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:13.901634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cca258-9b1e-4a99-9728-81a49eb16e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:13.901904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cca258-9b1e-4a99-9728-81a49eb16e97 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:13.902895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cca258-9b1e-4a99-9728-81a49eb16e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:13.903205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cca258-9b1e-4a99-9728-81a49eb16e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:13.905169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cca258-9b1e-4a99-9728-81a49eb16e97 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:13.917768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 587/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:19:13.926155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cc44c-b1c0-4308-b7cf-dc96c1b3e4ca None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:13.978124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:14.030873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cc44c-b1c0-4308-b7cf-dc96c1b3e4ca None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:14.070061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655cdaf-a430-4cb8-b894-7620193c2bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:14.070744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655cdaf-a430-4cb8-b894-7620193c2bd3 None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:14.071028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655cdaf-a430-4cb8-b894-7620193c2bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:14.071305 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655cdaf-a430-4cb8-b894-7620193c2bd3 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:14.072060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655cdaf-a430-4cb8-b894-7620193c2bd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:14.073599 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655cdaf-a430-4cb8-b894-7620193c2bd3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:14.264657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 618/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Mon Jun 27 09:19:13 2022] POST /identity/v3/users => generated 251 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:19:16.377332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656f7be-2a9f-4b8b-aad0-d799afbd5444 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:16.377878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656f7be-2a9f-4b8b-aad0-d799afbd5444 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:16.377982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656f7be-2a9f-4b8b-aad0-d799afbd5444 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:16.379331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 588/1206] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:16.385187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159008dc-aebb-4419-adb5-adaa2285b8b9 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:16.385713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159008dc-aebb-4419-adb5-adaa2285b8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:16.385822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159008dc-aebb-4419-adb5-adaa2285b8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:16.403739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-159008dc-aebb-4419-adb5-adaa2285b8b9 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:16.405540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-159008dc-aebb-4419-adb5-adaa2285b8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:16.474078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 619/1207] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:19:16.499159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656f7be-2a9f-4b8b-aad0-d799afbd5444 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:16.602865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656f7be-2a9f-4b8b-aad0-d799afbd5444 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:16.636189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da56408-c3c5-411f-988a-2e42d9c6312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:16.636985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da56408-c3c5-411f-988a-2e42d9c6312b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:16.637269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da56408-c3c5-411f-988a-2e42d9c6312b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:16.637542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da56408-c3c5-411f-988a-2e42d9c6312b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:16.648350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da56408-c3c5-411f-988a-2e42d9c6312b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-9da56408-c3c5-411f-988a-2e42d9c6312b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:19:16.657054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:16.662440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:16.663574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 589/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:19:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:16.664456 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159008dc-aebb-4419-adb5-adaa2285b8b9 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:16.704530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:16.771539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159008dc-aebb-4419-adb5-adaa2285b8b9 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:16.801162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff484-9fb5-4506-b47e-0022f62962dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:16.801896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff484-9fb5-4506-b47e-0022f62962dc None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:16.802178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff484-9fb5-4506-b47e-0022f62962dc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:16.802453 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff484-9fb5-4506-b47e-0022f62962dc None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:16.803203 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ff484-9fb5-4506-b47e-0022f62962dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:16.803509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ff484-9fb5-4506-b47e-0022f62962dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:16.804978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ff484-9fb5-4506-b47e-0022f62962dc None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:16.814777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 620/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:19:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:16.820600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da56408-c3c5-411f-988a-2e42d9c6312b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:16.871443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:16.927089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da56408-c3c5-411f-988a-2e42d9c6312b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:16.961737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63325b24-f335-4c63-86bb-3e9bd28d491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:16.962530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63325b24-f335-4c63-86bb-3e9bd28d491b None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:16.962815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63325b24-f335-4c63-86bb-3e9bd28d491b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:16.963095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63325b24-f335-4c63-86bb-3e9bd28d491b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:16.973654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63325b24-f335-4c63-86bb-3e9bd28d491b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:16.975103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63325b24-f335-4c63-86bb-3e9bd28d491b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-63325b24-f335-4c63-86bb-3e9bd28d491b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:16.992194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:16.993140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:16.994158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:16.994158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:16.994158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:16.994158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:16.994158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 590/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:17.001261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff484-9fb5-4506-b47e-0022f62962dc None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:17.095644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:17.098725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff484-9fb5-4506-b47e-0022f62962dc None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:17.133746 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895d05e-9bd7-47d3-be61-1a53890a0016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:17.134332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895d05e-9bd7-47d3-be61-1a53890a0016 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:17.134618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895d05e-9bd7-47d3-be61-1a53890a0016 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:17.134899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895d05e-9bd7-47d3-be61-1a53890a0016 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:17.135649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895d05e-9bd7-47d3-be61-1a53890a0016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:17.135968 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895d05e-9bd7-47d3-be61-1a53890a0016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:17.137331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895d05e-9bd7-47d3-be61-1a53890a0016 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:17.148876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 621/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:17.154977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63325b24-f335-4c63-86bb-3e9bd28d491b None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:17.256582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63325b24-f335-4c63-86bb-3e9bd28d491b None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:17.258126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:17.294283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643dddc-9692-43d7-9b33-879f10780d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:17.295715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643dddc-9692-43d7-9b33-879f10780d14 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:17.295998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643dddc-9692-43d7-9b33-879f10780d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:17.296277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643dddc-9692-43d7-9b33-879f10780d14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:17.304250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643dddc-9692-43d7-9b33-879f10780d14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:17.305733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643dddc-9692-43d7-9b33-879f10780d14 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-1643dddc-9692-43d7-9b33-879f10780d14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:19:17.322747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:17.328171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:17.329689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 591/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:17.332350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895d05e-9bd7-47d3-be61-1a53890a0016 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:17.448189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895d05e-9bd7-47d3-be61-1a53890a0016 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:17.481797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279a6a3-3449-490c-a4a6-fe8b3e77c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:17.483040 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279a6a3-3449-490c-a4a6-fe8b3e77c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:17.483313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279a6a3-3449-490c-a4a6-fe8b3e77c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:17.483606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279a6a3-3449-490c-a4a6-fe8b3e77c6c8 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:17.484480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279a6a3-3449-490c-a4a6-fe8b3e77c6c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:17.484786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279a6a3-3449-490c-a4a6-fe8b3e77c6c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:17.486221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279a6a3-3449-490c-a4a6-fe8b3e77c6c8 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:17.504986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 622/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Mon Jun 27 09:19:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:17.511065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643dddc-9692-43d7-9b33-879f10780d14 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:17.622984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643dddc-9692-43d7-9b33-879f10780d14 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:17.659626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299f17-1588-4ef1-9488-e823e1f1494f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:17.660293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299f17-1588-4ef1-9488-e823e1f1494f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:17.660575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299f17-1588-4ef1-9488-e823e1f1494f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:17.660879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299f17-1588-4ef1-9488-e823e1f1494f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:17.693926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:17.733608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8299f17-1588-4ef1-9488-e823e1f1494f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:17.735093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8299f17-1588-4ef1-9488-e823e1f1494f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:17.840744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 592/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:19:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 09:19:17.847346 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279a6a3-3449-490c-a4a6-fe8b3e77c6c8 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:17.867033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:17.952937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279a6a3-3449-490c-a4a6-fe8b3e77c6c8 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:17.990673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9f97b-48b9-4da4-ac35-465915089dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:17.991397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9f97b-48b9-4da4-ac35-465915089dca None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:17.991716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9f97b-48b9-4da4-ac35-465915089dca None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:17.991991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9f97b-48b9-4da4-ac35-465915089dca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:18.004024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9f97b-48b9-4da4-ac35-465915089dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:18.005566 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9f97b-48b9-4da4-ac35-465915089dca None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-abf9f97b-48b9-4da4-ac35-465915089dca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:18.025564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:18.027237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:18.030008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:18.030008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:18.030008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:18.030008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:18.030008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 623/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:18.034868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299f17-1588-4ef1-9488-e823e1f1494f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:18.098456 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:18.140478 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299f17-1588-4ef1-9488-e823e1f1494f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:18.175695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f404879-ee29-4357-ae51-0c8694a44318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:18.176447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f404879-ee29-4357-ae51-0c8694a44318 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:18.176727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f404879-ee29-4357-ae51-0c8694a44318 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:18.177005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f404879-ee29-4357-ae51-0c8694a44318 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:18.177737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f404879-ee29-4357-ae51-0c8694a44318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:18.178048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f404879-ee29-4357-ae51-0c8694a44318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:18.179515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f404879-ee29-4357-ae51-0c8694a44318 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:18.191603 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 593/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:18.197401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9f97b-48b9-4da4-ac35-465915089dca None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:18.269296 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:18.299523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9f97b-48b9-4da4-ac35-465915089dca None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:18.335803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b1ae1-8e8c-44d6-9c0d-0eab3ec85404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:18.336649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b1ae1-8e8c-44d6-9c0d-0eab3ec85404 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:18.336929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b1ae1-8e8c-44d6-9c0d-0eab3ec85404 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:18.337206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b1ae1-8e8c-44d6-9c0d-0eab3ec85404 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:18.344388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b1ae1-8e8c-44d6-9c0d-0eab3ec85404 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:18.345838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b1ae1-8e8c-44d6-9c0d-0eab3ec85404 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-9d6b1ae1-8e8c-44d6-9c0d-0eab3ec85404 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:19:18.359887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:18.361569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:18.362828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:19:18.362828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:19:18.362828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:19:18.362828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:19:18.362828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:19:18.362828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:19:18.362828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:18.362828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:18.362828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 624/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:19:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:18.367087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f404879-ee29-4357-ae51-0c8694a44318 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:18.472006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f404879-ee29-4357-ae51-0c8694a44318 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:18.496400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:18.516211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c73da5-8207-4245-859c-ca8888362923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:18.517246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c73da5-8207-4245-859c-ca8888362923 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:18.517677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c73da5-8207-4245-859c-ca8888362923 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:18.518115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c73da5-8207-4245-859c-ca8888362923 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:18.519677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c73da5-8207-4245-859c-ca8888362923 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:18.520115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c73da5-8207-4245-859c-ca8888362923 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:18.522517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c73da5-8207-4245-859c-ca8888362923 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:18.543953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 594/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:19:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:18.551279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b1ae1-8e8c-44d6-9c0d-0eab3ec85404 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:18.658027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b1ae1-8e8c-44d6-9c0d-0eab3ec85404 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:18.688563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:18.694680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea92d4-0691-40db-b9fa-1fb680ca489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:18.700069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea92d4-0691-40db-b9fa-1fb680ca489f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:18.700069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea92d4-0691-40db-b9fa-1fb680ca489f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:18.700069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea92d4-0691-40db-b9fa-1fb680ca489f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:18.700069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea92d4-0691-40db-b9fa-1fb680ca489f None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19d4d5fa82dc42e498fb22349921885e) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:18.700069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea92d4-0691-40db-b9fa-1fb680ca489f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:18.700069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea92d4-0691-40db-b9fa-1fb680ca489f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:18.704972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 625/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:19:18 2022] GET /identity/v3/role_assignments?user.id=19d4d5fa82dc42e498fb22349921885e&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:20.718596 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f30e0-7555-4688-aec8-26a90f1430cd None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:20.719092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f30e0-7555-4688-aec8-26a90f1430cd None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:20.719092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f30e0-7555-4688-aec8-26a90f1430cd None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:20.720577 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 595/1220] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:20.726460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1023-b2e3-41d3-b807-25a0d34dac08 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:20.726906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1023-b2e3-41d3-b807-25a0d34dac08 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:20.727069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1023-b2e3-41d3-b807-25a0d34dac08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:20.744377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d24a1023-b2e3-41d3-b807-25a0d34dac08 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:20.745616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d24a1023-b2e3-41d3-b807-25a0d34dac08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:20.813819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 626/1221] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:19:20.837430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f30e0-7555-4688-aec8-26a90f1430cd None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:20.943292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f30e0-7555-4688-aec8-26a90f1430cd None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:20.978246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7bcbb-0ff6-427d-8a1c-b4bc6641bf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:20.978980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7bcbb-0ff6-427d-8a1c-b4bc6641bf33 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:20.979266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7bcbb-0ff6-427d-8a1c-b4bc6641bf33 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:20.979556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7bcbb-0ff6-427d-8a1c-b4bc6641bf33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:20.989547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7bcbb-0ff6-427d-8a1c-b4bc6641bf33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-a8b7bcbb-0ff6-427d-8a1c-b4bc6641bf33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:19:20.997395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:20.998519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:20.999446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 596/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:19:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:21.005124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a1023-b2e3-41d3-b807-25a0d34dac08 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:21.112131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a1023-b2e3-41d3-b807-25a0d34dac08 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:21.136129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:21.147148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3f84e-5a5e-49cc-8375-92f359092e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:21.147908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3f84e-5a5e-49cc-8375-92f359092e51 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:21.148197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3f84e-5a5e-49cc-8375-92f359092e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:21.148474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3f84e-5a5e-49cc-8375-92f359092e51 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:21.149246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3f84e-5a5e-49cc-8375-92f359092e51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:21.149552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3f84e-5a5e-49cc-8375-92f359092e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:21.150947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3f84e-5a5e-49cc-8375-92f359092e51 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:21.160635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 627/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:19:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:21.166536 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7bcbb-0ff6-427d-8a1c-b4bc6641bf33 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:21.211358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:21.268680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7bcbb-0ff6-427d-8a1c-b4bc6641bf33 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:21.302407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713c8e7-7d5a-4ba7-93f0-1bd9bd0b36af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:21.303201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713c8e7-7d5a-4ba7-93f0-1bd9bd0b36af None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:21.303486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713c8e7-7d5a-4ba7-93f0-1bd9bd0b36af None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:21.303800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713c8e7-7d5a-4ba7-93f0-1bd9bd0b36af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:21.316891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713c8e7-7d5a-4ba7-93f0-1bd9bd0b36af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:21.318510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713c8e7-7d5a-4ba7-93f0-1bd9bd0b36af None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-3713c8e7-7d5a-4ba7-93f0-1bd9bd0b36af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:21.336962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:21.338040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:21.339151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:21.339151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:21.339151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:21.339151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:21.339151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 597/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:21.344332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3f84e-5a5e-49cc-8375-92f359092e51 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:21.445679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3f84e-5a5e-49cc-8375-92f359092e51 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:21.479334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6309c8b-c66e-49d1-9d4e-11ef061fa34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:21.480030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6309c8b-c66e-49d1-9d4e-11ef061fa34b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:21.480316 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6309c8b-c66e-49d1-9d4e-11ef061fa34b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:21.480593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6309c8b-c66e-49d1-9d4e-11ef061fa34b None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:21.481310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6309c8b-c66e-49d1-9d4e-11ef061fa34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:21.481618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6309c8b-c66e-49d1-9d4e-11ef061fa34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:21.483027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6309c8b-c66e-49d1-9d4e-11ef061fa34b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:21.495887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 628/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:21.503683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713c8e7-7d5a-4ba7-93f0-1bd9bd0b36af None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:21.604138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:21.611343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713c8e7-7d5a-4ba7-93f0-1bd9bd0b36af None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:21.646450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43d629-0951-4c1d-b638-36e58a230d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:21.647394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43d629-0951-4c1d-b638-36e58a230d1f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:21.647816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43d629-0951-4c1d-b638-36e58a230d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:21.648193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43d629-0951-4c1d-b638-36e58a230d1f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:21.655826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43d629-0951-4c1d-b638-36e58a230d1f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:21.657372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43d629-0951-4c1d-b638-36e58a230d1f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0e43d629-0951-4c1d-b638-36e58a230d1f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:19:21.672472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:21.673479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:21.674428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 598/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:21.679760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6309c8b-c66e-49d1-9d4e-11ef061fa34b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:21.727997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:21.781775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6309c8b-c66e-49d1-9d4e-11ef061fa34b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:21.817187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dacb6-5209-4282-b557-c99a9261afd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:21.817855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dacb6-5209-4282-b557-c99a9261afd3 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:21.818139 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dacb6-5209-4282-b557-c99a9261afd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:21.818547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dacb6-5209-4282-b557-c99a9261afd3 None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:21.819388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3dacb6-5209-4282-b557-c99a9261afd3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:21.819727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3dacb6-5209-4282-b557-c99a9261afd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:21.821163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3dacb6-5209-4282-b557-c99a9261afd3 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:21.839767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 629/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Mon Jun 27 09:19:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:21.845640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43d629-0951-4c1d-b638-36e58a230d1f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:21.949682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43d629-0951-4c1d-b638-36e58a230d1f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:21.984775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86d1d2-df1d-4d15-afb6-5f80c6d4aaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:21.985574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86d1d2-df1d-4d15-afb6-5f80c6d4aaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:21.985862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86d1d2-df1d-4d15-afb6-5f80c6d4aaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:21.986137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86d1d2-df1d-4d15-afb6-5f80c6d4aaa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:21.996847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86d1d2-df1d-4d15-afb6-5f80c6d4aaa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:22.000987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86d1d2-df1d-4d15-afb6-5f80c6d4aaa2 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-1f86d1d2-df1d-4d15-afb6-5f80c6d4aaa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:22.017668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:22.018777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:22.019724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:22.019724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:22.019724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:22.019724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:22.019724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 599/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:22.025123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dacb6-5209-4282-b557-c99a9261afd3 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:22.135847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dacb6-5209-4282-b557-c99a9261afd3 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:22.138212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:22.170735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7745cd-080e-4199-b4dd-3b453f1447b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:22.171396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7745cd-080e-4199-b4dd-3b453f1447b1 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:22.171712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7745cd-080e-4199-b4dd-3b453f1447b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:22.171992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7745cd-080e-4199-b4dd-3b453f1447b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:22.172715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7745cd-080e-4199-b4dd-3b453f1447b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:22.173023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7745cd-080e-4199-b4dd-3b453f1447b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:22.174396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7745cd-080e-4199-b4dd-3b453f1447b1 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:22.185808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 630/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:22.191718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86d1d2-df1d-4d15-afb6-5f80c6d4aaa2 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:22.201014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:22.299732 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86d1d2-df1d-4d15-afb6-5f80c6d4aaa2 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:22.335433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e286a-9d04-496c-a659-a7cd239d82e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:22.336327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e286a-9d04-496c-a659-a7cd239d82e6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:22.337574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e286a-9d04-496c-a659-a7cd239d82e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:22.337863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e286a-9d04-496c-a659-a7cd239d82e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:22.345665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e286a-9d04-496c-a659-a7cd239d82e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:22.347196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e286a-9d04-496c-a659-a7cd239d82e6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-a39e286a-9d04-496c-a659-a7cd239d82e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:19:22.362623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:22.363844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:22.365193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:19:22.365193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:19:22.365193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:19:22.365193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:19:22.365193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:19:22.365193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:19:22.365193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:22.365193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:22.365193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 600/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:19:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:22.370201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7745cd-080e-4199-b4dd-3b453f1447b1 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:22.470826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7745cd-080e-4199-b4dd-3b453f1447b1 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:22.504027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa44e3-c095-4924-b0d3-1d68e4213d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:22.504683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa44e3-c095-4924-b0d3-1d68e4213d68 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:22.504961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa44e3-c095-4924-b0d3-1d68e4213d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:22.505232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa44e3-c095-4924-b0d3-1d68e4213d68 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:22.506007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa44e3-c095-4924-b0d3-1d68e4213d68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:22.506308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa44e3-c095-4924-b0d3-1d68e4213d68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:22.507863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa44e3-c095-4924-b0d3-1d68e4213d68 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:22.520975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 631/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:19:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:19:22.527165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e286a-9d04-496c-a659-a7cd239d82e6 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:22.592928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:22.626394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e286a-9d04-496c-a659-a7cd239d82e6 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:22.662569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448ba50-bd20-4440-b0a2-3500f85ac4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:22.663459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448ba50-bd20-4440-b0a2-3500f85ac4a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:22.663772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448ba50-bd20-4440-b0a2-3500f85ac4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:22.664050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448ba50-bd20-4440-b0a2-3500f85ac4a0 None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19d4d5fa82dc42e498fb22349921885e/roles/ee8bde816f274202886522710bb8fdf2` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:22.680052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448ba50-bd20-4440-b0a2-3500f85ac4a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=19d4d5fa82dc42e498fb22349921885e, role_id=ee8bde816f274202886522710bb8fdf2)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:22.681533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448ba50-bd20-4440-b0a2-3500f85ac4a0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:22.804336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 601/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:19:22 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19d4d5fa82dc42e498fb22349921885e/roles/ee8bde816f274202886522710bb8fdf2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:19:24.843115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b4ec6-0459-43bb-80a2-acb963290f09 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:24.843663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b4ec6-0459-43bb-80a2-acb963290f09 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:24.843755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b4ec6-0459-43bb-80a2-acb963290f09 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:24.845130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 632/1233] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:24.850472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4af427-3528-496c-90e1-9f7ac6e2a9af None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:24.850854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4af427-3528-496c-90e1-9f7ac6e2a9af None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:24.851026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4af427-3528-496c-90e1-9f7ac6e2a9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:24.868065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-0f4af427-3528-496c-90e1-9f7ac6e2a9af None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:24.869335 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0f4af427-3528-496c-90e1-9f7ac6e2a9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:24.943426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 602/1234] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:19:24.966972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b4ec6-0459-43bb-80a2-acb963290f09 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:24.999000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:25.081645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b4ec6-0459-43bb-80a2-acb963290f09 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:25.118064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16708a7-8818-43d1-969f-809a5e53557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:25.118793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16708a7-8818-43d1-969f-809a5e53557a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:25.119071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16708a7-8818-43d1-969f-809a5e53557a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:25.119340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16708a7-8818-43d1-969f-809a5e53557a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:25.132251 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16708a7-8818-43d1-969f-809a5e53557a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-d16708a7-8818-43d1-969f-809a5e53557a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:19:25.140370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:25.141563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:25.143023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 633/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Mon Jun 27 09:19:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:25.148404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4af427-3528-496c-90e1-9f7ac6e2a9af None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:25.199103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:25.282397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4af427-3528-496c-90e1-9f7ac6e2a9af None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:25.320494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a48e1a-489a-4196-be7f-d5f04e2dd418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:25.321169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a48e1a-489a-4196-be7f-d5f04e2dd418 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:25.321439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a48e1a-489a-4196-be7f-d5f04e2dd418 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:25.321719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a48e1a-489a-4196-be7f-d5f04e2dd418 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:25.322469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a48e1a-489a-4196-be7f-d5f04e2dd418 None admin] RBAC: Adding query filter params (name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:25.322774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a48e1a-489a-4196-be7f-d5f04e2dd418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:25.324195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a48e1a-489a-4196-be7f-d5f04e2dd418 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:25.332752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 603/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Mon Jun 27 09:19:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:25.338843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16708a7-8818-43d1-969f-809a5e53557a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:25.429097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:25.444281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16708a7-8818-43d1-969f-809a5e53557a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:25.478839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e654d9-edf5-40c0-8424-eea7d073939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:25.479715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e654d9-edf5-40c0-8424-eea7d073939a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:25.479998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e654d9-edf5-40c0-8424-eea7d073939a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:25.480275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e654d9-edf5-40c0-8424-eea7d073939a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:25.491268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e654d9-edf5-40c0-8424-eea7d073939a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:25.495904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e654d9-edf5-40c0-8424-eea7d073939a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-00e654d9-edf5-40c0-8424-eea7d073939a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:25.511434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:25.512580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:25.514584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:25.514584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:25.514584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:25.514584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:25.514584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 634/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:25.519078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a48e1a-489a-4196-be7f-d5f04e2dd418 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:25.621847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a48e1a-489a-4196-be7f-d5f04e2dd418 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:25.632177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:25.658659 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8837f-6bfa-4026-821c-515f9eded1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:25.659336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8837f-6bfa-4026-821c-515f9eded1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:25.659650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8837f-6bfa-4026-821c-515f9eded1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:25.659928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8837f-6bfa-4026-821c-515f9eded1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:25.660670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8837f-6bfa-4026-821c-515f9eded1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:25.660978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8837f-6bfa-4026-821c-515f9eded1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:25.662362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d8837f-6bfa-4026-821c-515f9eded1c2 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:25.675010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 604/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:25.681150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e654d9-edf5-40c0-8424-eea7d073939a None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:25.788980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e654d9-edf5-40c0-8424-eea7d073939a None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:25.824055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a30e5-5e49-4285-a589-3ae40cd2c58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:25.824877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a30e5-5e49-4285-a589-3ae40cd2c58b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:25.825160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a30e5-5e49-4285-a589-3ae40cd2c58b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:25.825438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a30e5-5e49-4285-a589-3ae40cd2c58b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:25.829154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:25.834463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a30e5-5e49-4285-a589-3ae40cd2c58b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:25.835969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a30e5-5e49-4285-a589-3ae40cd2c58b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-c69a30e5-5e49-4285-a589-3ae40cd2c58b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:19:25.851029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:25.852843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:25.854077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 635/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:25.859255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8837f-6bfa-4026-821c-515f9eded1c2 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:25.961451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8837f-6bfa-4026-821c-515f9eded1c2 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:25.995565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a7820-75c3-4a0d-b7da-c1707a52b556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:25.996251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a7820-75c3-4a0d-b7da-c1707a52b556 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:25.996528 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a7820-75c3-4a0d-b7da-c1707a52b556 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:25.996800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a7820-75c3-4a0d-b7da-c1707a52b556 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:25.998128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a7820-75c3-4a0d-b7da-c1707a52b556 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:25.998480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a7820-75c3-4a0d-b7da-c1707a52b556 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:26.000008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a7820-75c3-4a0d-b7da-c1707a52b556 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:26.020397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 605/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Mon Jun 27 09:19:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:26.026380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a30e5-5e49-4285-a589-3ae40cd2c58b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:26.129637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a30e5-5e49-4285-a589-3ae40cd2c58b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:26.162391 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3dd0d-f3fa-4984-b321-52d9b04db624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:26.163043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3dd0d-f3fa-4984-b321-52d9b04db624 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:26.163317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3dd0d-f3fa-4984-b321-52d9b04db624 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:26.163615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3dd0d-f3fa-4984-b321-52d9b04db624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:26.227307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:26.234376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3dd0d-f3fa-4984-b321-52d9b04db624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:26.235863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3dd0d-f3fa-4984-b321-52d9b04db624 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:26.350834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 636/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:19:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:19:26.357510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a7820-75c3-4a0d-b7da-c1707a52b556 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:26.387895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:26.463810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a7820-75c3-4a0d-b7da-c1707a52b556 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:26.500520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29907b-5ce2-4956-ac2e-e14209e63a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:26.501324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29907b-5ce2-4956-ac2e-e14209e63a31 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:26.501608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29907b-5ce2-4956-ac2e-e14209e63a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:26.501920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29907b-5ce2-4956-ac2e-e14209e63a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:26.512340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29907b-5ce2-4956-ac2e-e14209e63a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:26.513917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29907b-5ce2-4956-ac2e-e14209e63a31 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-9d29907b-5ce2-4956-ac2e-e14209e63a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:26.533880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:26.535189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:26.536779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:26.536779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:26.536779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:26.536779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:26.536779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 606/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:26.541054 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3dd0d-f3fa-4984-b321-52d9b04db624 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:26.638864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:26.644947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3dd0d-f3fa-4984-b321-52d9b04db624 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:26.690523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522883b-2636-455b-b8a5-c74b60bc521f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:26.691224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522883b-2636-455b-b8a5-c74b60bc521f None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:26.691509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522883b-2636-455b-b8a5-c74b60bc521f None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:26.691822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522883b-2636-455b-b8a5-c74b60bc521f None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:26.692552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522883b-2636-455b-b8a5-c74b60bc521f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:26.692860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522883b-2636-455b-b8a5-c74b60bc521f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:26.694262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522883b-2636-455b-b8a5-c74b60bc521f None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:26.707126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 637/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:26.717254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29907b-5ce2-4956-ac2e-e14209e63a31 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:26.806116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:26.827784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29907b-5ce2-4956-ac2e-e14209e63a31 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:26.865788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7f79-84a7-46e8-add7-efeaa112ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:26.866588 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7f79-84a7-46e8-add7-efeaa112ee3c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:26.866917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7f79-84a7-46e8-add7-efeaa112ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:26.867199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7f79-84a7-46e8-add7-efeaa112ee3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:26.874200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e7f79-84a7-46e8-add7-efeaa112ee3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:26.878439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e7f79-84a7-46e8-add7-efeaa112ee3c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-276e7f79-84a7-46e8-add7-efeaa112ee3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:19:26.890879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:26.892369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:26.893636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:19:26.893636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:19:26.893636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:19:26.893636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:19:26.893636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:19:26.893636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:19:26.893636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:26.893636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:26.893636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 607/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:19:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:26.898120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522883b-2636-455b-b8a5-c74b60bc521f None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:27.004308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522883b-2636-455b-b8a5-c74b60bc521f None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:27.040963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922e051-9d96-4cc1-ad57-dd36ae083285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:27.041624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922e051-9d96-4cc1-ad57-dd36ae083285 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:27.041908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922e051-9d96-4cc1-ad57-dd36ae083285 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:27.042185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922e051-9d96-4cc1-ad57-dd36ae083285 None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:27.042975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922e051-9d96-4cc1-ad57-dd36ae083285 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:27.043281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922e051-9d96-4cc1-ad57-dd36ae083285 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:27.044792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922e051-9d96-4cc1-ad57-dd36ae083285 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:27.057209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 638/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:19:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:27.063465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7f79-84a7-46e8-add7-efeaa112ee3c None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:27.162146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7f79-84a7-46e8-add7-efeaa112ee3c None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:27.194547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da26a76-302b-4445-b25c-ed2d26befc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:27.195195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da26a76-302b-4445-b25c-ed2d26befc63 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:27.195468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da26a76-302b-4445-b25c-ed2d26befc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:27.195771 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da26a76-302b-4445-b25c-ed2d26befc63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:27.196594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da26a76-302b-4445-b25c-ed2d26befc63 None admin] RBAC: Adding query filter params (role.id=ee8bde816f274202886522710bb8fdf2, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19d4d5fa82dc42e498fb22349921885e) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:27.196909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da26a76-302b-4445-b25c-ed2d26befc63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:27.198324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da26a76-302b-4445-b25c-ed2d26befc63 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:27.204689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 608/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:19:27 2022] GET /identity/v3/role_assignments?user.id=19d4d5fa82dc42e498fb22349921885e&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=ee8bde816f274202886522710bb8fdf2 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:29.331732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba844e9-cecf-4411-b564-88f72d5c755e None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:29.332255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba844e9-cecf-4411-b564-88f72d5c755e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:29.332356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba844e9-cecf-4411-b564-88f72d5c755e None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:29.333705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 639/1247] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:29.338851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61c369-c282-46aa-a669-468b53291cdd None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:29.343387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61c369-c282-46aa-a669-468b53291cdd None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:29.343387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61c369-c282-46aa-a669-468b53291cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:29.344590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:29.356440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4f61c369-c282-46aa-a669-468b53291cdd None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:29.358150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4f61c369-c282-46aa-a669-468b53291cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:29.425665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 609/1248] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:19:29.449774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba844e9-cecf-4411-b564-88f72d5c755e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:29.467568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:29.553548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba844e9-cecf-4411-b564-88f72d5c755e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:29.586106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e8dcd-dddd-4463-832c-e4ce18ad88c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:29.586894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e8dcd-dddd-4463-832c-e4ce18ad88c8 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:29.587176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e8dcd-dddd-4463-832c-e4ce18ad88c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:29.587471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e8dcd-dddd-4463-832c-e4ce18ad88c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:29.598113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e8dcd-dddd-4463-832c-e4ce18ad88c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-b52e8dcd-dddd-4463-832c-e4ce18ad88c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:19:29.605928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:29.609561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:29.610729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 640/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:19:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:29.613645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61c369-c282-46aa-a669-468b53291cdd None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:29.714500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61c369-c282-46aa-a669-468b53291cdd None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:29.744717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:29.749801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4cbda-8c96-42ee-85fc-82bc740bed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:29.750476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4cbda-8c96-42ee-85fc-82bc740bed88 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:29.750755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4cbda-8c96-42ee-85fc-82bc740bed88 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:29.751027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4cbda-8c96-42ee-85fc-82bc740bed88 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:29.751807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4cbda-8c96-42ee-85fc-82bc740bed88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:29.752170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4cbda-8c96-42ee-85fc-82bc740bed88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:29.753535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4cbda-8c96-42ee-85fc-82bc740bed88 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:29.761646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 610/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:19:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:29.767410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e8dcd-dddd-4463-832c-e4ce18ad88c8 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:29.844704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:29.868066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e8dcd-dddd-4463-832c-e4ce18ad88c8 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:29.903225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efc8c4-8436-491f-9aaa-f6be47598ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:29.904601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efc8c4-8436-491f-9aaa-f6be47598ddc None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:29.904878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efc8c4-8436-491f-9aaa-f6be47598ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:29.905148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efc8c4-8436-491f-9aaa-f6be47598ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:29.916147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efc8c4-8436-491f-9aaa-f6be47598ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:29.921693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efc8c4-8436-491f-9aaa-f6be47598ddc None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-71efc8c4-8436-491f-9aaa-f6be47598ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:29.935683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:29.936970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:29.937977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:29.937977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:29.937977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:29.937977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:29.937977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 641/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:29.943496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4cbda-8c96-42ee-85fc-82bc740bed88 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:30.050107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4cbda-8c96-42ee-85fc-82bc740bed88 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:30.084965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d163-2f99-4cdd-a2f6-0481a0c10180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:30.085628 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d163-2f99-4cdd-a2f6-0481a0c10180 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:30.085906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d163-2f99-4cdd-a2f6-0481a0c10180 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:30.086178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506d163-2f99-4cdd-a2f6-0481a0c10180 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:30.086898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506d163-2f99-4cdd-a2f6-0481a0c10180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:30.087235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506d163-2f99-4cdd-a2f6-0481a0c10180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:30.088684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506d163-2f99-4cdd-a2f6-0481a0c10180 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:30.104499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 611/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:30.107877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efc8c4-8436-491f-9aaa-f6be47598ddc None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:30.208050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efc8c4-8436-491f-9aaa-f6be47598ddc None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:30.234445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:30.242200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af77388-f317-4840-af8b-f378bde82416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:30.243002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af77388-f317-4840-af8b-f378bde82416 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:30.243284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af77388-f317-4840-af8b-f378bde82416 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:30.243587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af77388-f317-4840-af8b-f378bde82416 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:30.251760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af77388-f317-4840-af8b-f378bde82416 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:30.253162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af77388-f317-4840-af8b-f378bde82416 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3af77388-f317-4840-af8b-f378bde82416 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:19:30.269438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:30.277359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:30.279289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 642/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:30.279976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d163-2f99-4cdd-a2f6-0481a0c10180 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:30.335716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:30.380025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506d163-2f99-4cdd-a2f6-0481a0c10180 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:30.419286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c610305-4d33-400e-ab91-a82b03b51652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:30.419998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c610305-4d33-400e-ab91-a82b03b51652 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:30.420283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c610305-4d33-400e-ab91-a82b03b51652 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:30.420559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c610305-4d33-400e-ab91-a82b03b51652 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:30.421425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c610305-4d33-400e-ab91-a82b03b51652 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:30.421757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c610305-4d33-400e-ab91-a82b03b51652 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:30.423226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c610305-4d33-400e-ab91-a82b03b51652 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:30.443767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 612/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Mon Jun 27 09:19:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:30.450145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af77388-f317-4840-af8b-f378bde82416 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:30.553091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af77388-f317-4840-af8b-f378bde82416 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:30.587063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ade93d-f021-4586-a16d-054d0abb3b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:30.587744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ade93d-f021-4586-a16d-054d0abb3b3e None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:30.588026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ade93d-f021-4586-a16d-054d0abb3b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:30.588305 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ade93d-f021-4586-a16d-054d0abb3b3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:30.643414 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:30.655491 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ade93d-f021-4586-a16d-054d0abb3b3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:30.657145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ade93d-f021-4586-a16d-054d0abb3b3e None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:30.754837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 643/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:19:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:19:30.760968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c610305-4d33-400e-ab91-a82b03b51652 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:30.861307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c610305-4d33-400e-ab91-a82b03b51652 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:30.875811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:30.896278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148608-2a99-4128-86db-c7dda204da28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:30.897007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148608-2a99-4128-86db-c7dda204da28 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:30.897292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148608-2a99-4128-86db-c7dda204da28 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:30.897563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148608-2a99-4128-86db-c7dda204da28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:30.908544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b148608-2a99-4128-86db-c7dda204da28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:30.910008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b148608-2a99-4128-86db-c7dda204da28 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-0b148608-2a99-4128-86db-c7dda204da28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:30.928948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:30.931146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:30.932909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:30.932909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:30.932909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:30.932909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:30.932909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 613/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:30.936896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ade93d-f021-4586-a16d-054d0abb3b3e None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:31.031599 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:31.043843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ade93d-f021-4586-a16d-054d0abb3b3e None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:31.079804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723adef-349b-4021-83a1-04ccb2d11c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:31.084470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723adef-349b-4021-83a1-04ccb2d11c5b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:31.084470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723adef-349b-4021-83a1-04ccb2d11c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:31.084470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723adef-349b-4021-83a1-04ccb2d11c5b None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:31.084470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723adef-349b-4021-83a1-04ccb2d11c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:31.084470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723adef-349b-4021-83a1-04ccb2d11c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:31.084470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723adef-349b-4021-83a1-04ccb2d11c5b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:31.095677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 644/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:31.101704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148608-2a99-4128-86db-c7dda204da28 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:31.222015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148608-2a99-4128-86db-c7dda204da28 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:31.257037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5207d0-3b77-4d04-bfc1-907e4c0c946f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:31.257829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5207d0-3b77-4d04-bfc1-907e4c0c946f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:31.258166 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5207d0-3b77-4d04-bfc1-907e4c0c946f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:31.258388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5207d0-3b77-4d04-bfc1-907e4c0c946f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:31.265454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5207d0-3b77-4d04-bfc1-907e4c0c946f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:31.267391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5207d0-3b77-4d04-bfc1-907e4c0c946f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-ee5207d0-3b77-4d04-bfc1-907e4c0c946f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:19:31.282171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:31.283361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:31.284404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:19:31.284404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:19:31.284404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:19:31.284404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:19:31.284404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:19:31.284404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:19:31.284404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:31.284404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:31.284404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 614/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:19:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:31.289855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723adef-349b-4021-83a1-04ccb2d11c5b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:31.393952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723adef-349b-4021-83a1-04ccb2d11c5b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:31.428354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3c199-5b96-4166-8b5c-ffa921d5a48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:31.429018 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3c199-5b96-4166-8b5c-ffa921d5a48c None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:31.429302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3c199-5b96-4166-8b5c-ffa921d5a48c None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:31.429581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3c199-5b96-4166-8b5c-ffa921d5a48c None admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:31.430371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3c199-5b96-4166-8b5c-ffa921d5a48c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:31.430678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3c199-5b96-4166-8b5c-ffa921d5a48c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:31.432219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3c199-5b96-4166-8b5c-ffa921d5a48c None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:31.444434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 645/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:19:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:31.451351 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5207d0-3b77-4d04-bfc1-907e4c0c946f None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:31.484164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:31.561767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5207d0-3b77-4d04-bfc1-907e4c0c946f None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:31.597509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b43f61-cef7-4a35-a1e8-4084f28b34a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:31.598169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43f61-cef7-4a35-a1e8-4084f28b34a5 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:31.598455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43f61-cef7-4a35-a1e8-4084f28b34a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:31.598881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43f61-cef7-4a35-a1e8-4084f28b34a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:31.599764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43f61-cef7-4a35-a1e8-4084f28b34a5 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19d4d5fa82dc42e498fb22349921885e) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:31.600082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43f61-cef7-4a35-a1e8-4084f28b34a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:31.601530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43f61-cef7-4a35-a1e8-4084f28b34a5 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:31.607563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 615/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:19:31 2022] GET /identity/v3/role_assignments?user.id=19d4d5fa82dc42e498fb22349921885e&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=03b724e03f704f2ab338b64025a3db6d => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:33.686638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a07f43-7a69-4519-b4f8-3c3c750fc96d None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:33.687146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a07f43-7a69-4519-b4f8-3c3c750fc96d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:33.687146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a07f43-7a69-4519-b4f8-3c3c750fc96d None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:33.688556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 646/1261] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:33.694092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dce257-83be-4810-8c27-c7db6013f34c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:33.694564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dce257-83be-4810-8c27-c7db6013f34c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:33.694662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dce257-83be-4810-8c27-c7db6013f34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:33.713012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-12dce257-83be-4810-8c27-c7db6013f34c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:33.714228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-12dce257-83be-4810-8c27-c7db6013f34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:33.783283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 616/1262] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:19:33.806388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a07f43-7a69-4519-b4f8-3c3c750fc96d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:33.817591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:33.909565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a07f43-7a69-4519-b4f8-3c3c750fc96d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:33.945256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192a022-01d9-415a-9985-e22ecd57149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:33.946037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192a022-01d9-415a-9985-e22ecd57149b None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:33.946322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192a022-01d9-415a-9985-e22ecd57149b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:33.946653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192a022-01d9-415a-9985-e22ecd57149b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:33.958019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192a022-01d9-415a-9985-e22ecd57149b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-b192a022-01d9-415a-9985-e22ecd57149b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:19:33.966740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:33.967915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:33.968792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 647/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:19:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:33.974334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dce257-83be-4810-8c27-c7db6013f34c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:33.993281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:34.083171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dce257-83be-4810-8c27-c7db6013f34c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:34.118690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8467d1-630a-4177-813f-71222c556fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:34.119411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8467d1-630a-4177-813f-71222c556fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:34.119726 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8467d1-630a-4177-813f-71222c556fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:34.120233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8467d1-630a-4177-813f-71222c556fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:34.121061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8467d1-630a-4177-813f-71222c556fc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:34.125361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8467d1-630a-4177-813f-71222c556fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:34.125361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8467d1-630a-4177-813f-71222c556fc8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:34.133111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 617/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:19:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:34.139768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192a022-01d9-415a-9985-e22ecd57149b None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:34.208980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:34.242858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192a022-01d9-415a-9985-e22ecd57149b None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:34.277261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b180b87-0110-42e5-a499-13bc95dfc900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:34.278308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b180b87-0110-42e5-a499-13bc95dfc900 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:34.278660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b180b87-0110-42e5-a499-13bc95dfc900 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:34.278934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b180b87-0110-42e5-a499-13bc95dfc900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:34.290426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b180b87-0110-42e5-a499-13bc95dfc900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:34.291926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b180b87-0110-42e5-a499-13bc95dfc900 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-5b180b87-0110-42e5-a499-13bc95dfc900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:34.309356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:34.310974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:34.321761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:34.321761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:34.321761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:34.321761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:34.321761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 648/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:34.322128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8467d1-630a-4177-813f-71222c556fc8 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:34.397855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:34.416532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8467d1-630a-4177-813f-71222c556fc8 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:34.452568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d367-0ea7-47e9-bbe1-9d479abc75db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:34.453293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d367-0ea7-47e9-bbe1-9d479abc75db None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:34.453576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d367-0ea7-47e9-bbe1-9d479abc75db None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:34.453852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d367-0ea7-47e9-bbe1-9d479abc75db None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:34.454576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675d367-0ea7-47e9-bbe1-9d479abc75db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:34.454882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675d367-0ea7-47e9-bbe1-9d479abc75db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:34.456316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675d367-0ea7-47e9-bbe1-9d479abc75db None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:34.468244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 618/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:34.473919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b180b87-0110-42e5-a499-13bc95dfc900 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:34.573961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b180b87-0110-42e5-a499-13bc95dfc900 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:34.592948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:34.607052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a758de6-4157-41b9-8d1a-d28e52c046b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:34.607906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a758de6-4157-41b9-8d1a-d28e52c046b6 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:34.608189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a758de6-4157-41b9-8d1a-d28e52c046b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:34.608468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a758de6-4157-41b9-8d1a-d28e52c046b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:34.616105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a758de6-4157-41b9-8d1a-d28e52c046b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:34.617560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a758de6-4157-41b9-8d1a-d28e52c046b6 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-6a758de6-4157-41b9-8d1a-d28e52c046b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:19:34.639777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:34.641154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:34.642275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 649/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 09:19:34.646950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d367-0ea7-47e9-bbe1-9d479abc75db None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:34.751056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d367-0ea7-47e9-bbe1-9d479abc75db None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:34.787077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17daf02-4d3c-4d5b-8ab2-6ad89ad6c9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:34.787893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17daf02-4d3c-4d5b-8ab2-6ad89ad6c9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:34.788179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17daf02-4d3c-4d5b-8ab2-6ad89ad6c9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:34.788457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17daf02-4d3c-4d5b-8ab2-6ad89ad6c9b6 None admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:34.789307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17daf02-4d3c-4d5b-8ab2-6ad89ad6c9b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:34.789621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17daf02-4d3c-4d5b-8ab2-6ad89ad6c9b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:34.791069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17daf02-4d3c-4d5b-8ab2-6ad89ad6c9b6 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:34.801090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:34.809047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 619/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Mon Jun 27 09:19:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:34.814775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a758de6-4157-41b9-8d1a-d28e52c046b6 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:34.915341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a758de6-4157-41b9-8d1a-d28e52c046b6 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:34.949990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71da9d-fa4c-41b7-b8e6-672a7fe50944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:34.950806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71da9d-fa4c-41b7-b8e6-672a7fe50944 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:34.951089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71da9d-fa4c-41b7-b8e6-672a7fe50944 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:34.951365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71da9d-fa4c-41b7-b8e6-672a7fe50944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:34.961954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71da9d-fa4c-41b7-b8e6-672a7fe50944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:34.963573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71da9d-fa4c-41b7-b8e6-672a7fe50944 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-bb71da9d-fa4c-41b7-b8e6-672a7fe50944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:34.981351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:34.988310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:34.989430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:34.989430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:34.989430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:34.989430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:34.989430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 650/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:34.989787 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17daf02-4d3c-4d5b-8ab2-6ad89ad6c9b6 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:35.089663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17daf02-4d3c-4d5b-8ab2-6ad89ad6c9b6 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:35.126265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db37817-8e5f-4ac7-9759-6435dacf933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:35.126940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db37817-8e5f-4ac7-9759-6435dacf933a None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:35.127219 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db37817-8e5f-4ac7-9759-6435dacf933a None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:35.127500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db37817-8e5f-4ac7-9759-6435dacf933a None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:35.128277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db37817-8e5f-4ac7-9759-6435dacf933a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:35.128583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db37817-8e5f-4ac7-9759-6435dacf933a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:35.130020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db37817-8e5f-4ac7-9759-6435dacf933a None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:35.144111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 620/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:35.149854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71da9d-fa4c-41b7-b8e6-672a7fe50944 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:35.182927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:35.256869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71da9d-fa4c-41b7-b8e6-672a7fe50944 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:35.302976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdff445-c3cb-465c-804f-b98c65155350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:35.303856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdff445-c3cb-465c-804f-b98c65155350 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:35.304133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdff445-c3cb-465c-804f-b98c65155350 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:35.304406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdff445-c3cb-465c-804f-b98c65155350 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:35.314519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdff445-c3cb-465c-804f-b98c65155350 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:35.316018 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdff445-c3cb-465c-804f-b98c65155350 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3fdff445-c3cb-465c-804f-b98c65155350 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:19:35.331147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:35.332254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:35.333167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:19:35.333167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:19:35.333167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:19:35.333167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:19:35.333167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:19:35.333167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:19:35.333167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:35.333167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:35.333167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 651/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:19:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:35.338554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db37817-8e5f-4ac7-9759-6435dacf933a None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:35.399664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:35.440713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db37817-8e5f-4ac7-9759-6435dacf933a None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:35.474744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3eb6aa-2bd8-49ee-8384-429ce8d3bac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:35.475400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eb6aa-2bd8-49ee-8384-429ce8d3bac8 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:35.475716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eb6aa-2bd8-49ee-8384-429ce8d3bac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:35.475997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eb6aa-2bd8-49ee-8384-429ce8d3bac8 None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:35.476816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eb6aa-2bd8-49ee-8384-429ce8d3bac8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:35.477127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eb6aa-2bd8-49ee-8384-429ce8d3bac8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:35.478584 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eb6aa-2bd8-49ee-8384-429ce8d3bac8 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:35.491167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 621/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:19:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:35.497686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdff445-c3cb-465c-804f-b98c65155350 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:35.588074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:35.596340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdff445-c3cb-465c-804f-b98c65155350 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:35.629918 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb74c2-b6f3-46be-bd1e-3d48e672a91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:35.630800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb74c2-b6f3-46be-bd1e-3d48e672a91b None admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:35.631081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb74c2-b6f3-46be-bd1e-3d48e672a91b None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:35.631357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb74c2-b6f3-46be-bd1e-3d48e672a91b None admin] PATH_INFO: `/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19d4d5fa82dc42e498fb22349921885e/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:35.633437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb74c2-b6f3-46be-bd1e-3d48e672a91b None admin] RBAC: Authorizing `identity:create_grant(project_id=84f2268a6c4c456293cdde00cf0026be, user_id=19d4d5fa82dc42e498fb22349921885e, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:35.634867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb74c2-b6f3-46be-bd1e-3d48e672a91b None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:35.719179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 652/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Mon Jun 27 09:19:35 2022] PUT /identity/v3/projects/84f2268a6c4c456293cdde00cf0026be/users/19d4d5fa82dc42e498fb22349921885e/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:19:37.769256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6661c2-6356-461f-8891-875888211e91 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:37.769796 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6661c2-6356-461f-8891-875888211e91 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:37.769796 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6661c2-6356-461f-8891-875888211e91 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:37.771262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 622/1274] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:37.776693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe88fc4-1a9a-4ccd-a299-aea96d465912 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:37.777170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe88fc4-1a9a-4ccd-a299-aea96d465912 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:37.777262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe88fc4-1a9a-4ccd-a299-aea96d465912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:37.795264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ebe88fc4-1a9a-4ccd-a299-aea96d465912 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:37.796509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ebe88fc4-1a9a-4ccd-a299-aea96d465912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:37.870389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 653/1275] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:19:37.893933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6661c2-6356-461f-8891-875888211e91 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:37.997019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6661c2-6356-461f-8891-875888211e91 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:38.014533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:38.032100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dafa7f-6f68-479c-9fdf-c2870020fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:38.032802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dafa7f-6f68-479c-9fdf-c2870020fe2d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:38.033076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dafa7f-6f68-479c-9fdf-c2870020fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:38.033349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dafa7f-6f68-479c-9fdf-c2870020fe2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:38.043777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dafa7f-6f68-479c-9fdf-c2870020fe2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-e4dafa7f-6f68-479c-9fdf-c2870020fe2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 09:19:38.052248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:38.053374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:38.054286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 623/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Mon Jun 27 09:19:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:38.059616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe88fc4-1a9a-4ccd-a299-aea96d465912 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:38.145799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:38.165467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe88fc4-1a9a-4ccd-a299-aea96d465912 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:38.201844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02104f-4ab6-4354-9c00-b7dd79c0f769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:38.202568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02104f-4ab6-4354-9c00-b7dd79c0f769 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:38.202851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02104f-4ab6-4354-9c00-b7dd79c0f769 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:38.203127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02104f-4ab6-4354-9c00-b7dd79c0f769 None admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:38.203926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02104f-4ab6-4354-9c00-b7dd79c0f769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:38.204236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02104f-4ab6-4354-9c00-b7dd79c0f769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:38.205687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02104f-4ab6-4354-9c00-b7dd79c0f769 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:38.215052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 654/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Mon Jun 27 09:19:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:38.221811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dafa7f-6f68-479c-9fdf-c2870020fe2d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:38.338003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dafa7f-6f68-479c-9fdf-c2870020fe2d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:38.378318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61850fe-9ee7-4c3a-8491-336787b55693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:38.379129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61850fe-9ee7-4c3a-8491-336787b55693 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:38.379428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61850fe-9ee7-4c3a-8491-336787b55693 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:38.379735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61850fe-9ee7-4c3a-8491-336787b55693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:38.392267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61850fe-9ee7-4c3a-8491-336787b55693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:38.393745 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61850fe-9ee7-4c3a-8491-336787b55693 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-d61850fe-9ee7-4c3a-8491-336787b55693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:38.413123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:38.414163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:38.415052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:38.415052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:38.415052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:38.415052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:38.415052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 624/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:38.420912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02104f-4ab6-4354-9c00-b7dd79c0f769 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:38.532097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02104f-4ab6-4354-9c00-b7dd79c0f769 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:38.566979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5c05-1d5e-494b-838a-6a708444c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:38.568029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5c05-1d5e-494b-838a-6a708444c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:38.568541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5c05-1d5e-494b-838a-6a708444c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:38.569037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5c05-1d5e-494b-838a-6a708444c2d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:38.570428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5c05-1d5e-494b-838a-6a708444c2d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:38.570990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5c05-1d5e-494b-838a-6a708444c2d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:38.573668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5c05-1d5e-494b-838a-6a708444c2d7 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:38.596898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 655/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:19:38.602820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61850fe-9ee7-4c3a-8491-336787b55693 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:38.637986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:38.719981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61850fe-9ee7-4c3a-8491-336787b55693 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:38.757872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09902c-5e85-4d26-92a2-d9873f71849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:38.758803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09902c-5e85-4d26-92a2-d9873f71849d None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:38.759085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09902c-5e85-4d26-92a2-d9873f71849d None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:38.759361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09902c-5e85-4d26-92a2-d9873f71849d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:38.767765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09902c-5e85-4d26-92a2-d9873f71849d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:38.769241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09902c-5e85-4d26-92a2-d9873f71849d None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-5e09902c-5e85-4d26-92a2-d9873f71849d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 09:19:38.786278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:38.788347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:38.789498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 625/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:38.794265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5c05-1d5e-494b-838a-6a708444c2d7 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:38.798767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:38.915366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5c05-1d5e-494b-838a-6a708444c2d7 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:38.954161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36420c-75b8-4b92-a9b5-3f51f25771fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:38.954852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36420c-75b8-4b92-a9b5-3f51f25771fc None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:38.955132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36420c-75b8-4b92-a9b5-3f51f25771fc None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:38.955404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36420c-75b8-4b92-a9b5-3f51f25771fc None admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:38.956293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36420c-75b8-4b92-a9b5-3f51f25771fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:38.956600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36420c-75b8-4b92-a9b5-3f51f25771fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:38.958122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36420c-75b8-4b92-a9b5-3f51f25771fc None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:38.977114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 656/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Mon Jun 27 09:19:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:38.983477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09902c-5e85-4d26-92a2-d9873f71849d None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:39.078941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:39.088513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09902c-5e85-4d26-92a2-d9873f71849d None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:39.129119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83aeec-f57c-43c5-9f4d-97720717ed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:39.129827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83aeec-f57c-43c5-9f4d-97720717ed96 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:39.130113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83aeec-f57c-43c5-9f4d-97720717ed96 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:39.130389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83aeec-f57c-43c5-9f4d-97720717ed96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:39.206058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83aeec-f57c-43c5-9f4d-97720717ed96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:39.207481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83aeec-f57c-43c5-9f4d-97720717ed96 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:39.317918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:39.328323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 626/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Mon Jun 27 09:19:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 09:19:39.334372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36420c-75b8-4b92-a9b5-3f51f25771fc None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:39.399341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:39.443337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36420c-75b8-4b92-a9b5-3f51f25771fc None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:39.479048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0051a61-4df7-4398-a699-6f0233f5a1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:39.479795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0051a61-4df7-4398-a699-6f0233f5a1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:39.480076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0051a61-4df7-4398-a699-6f0233f5a1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:39.480355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0051a61-4df7-4398-a699-6f0233f5a1c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:39.492087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0051a61-4df7-4398-a699-6f0233f5a1c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:39.493559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0051a61-4df7-4398-a699-6f0233f5a1c4 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-c0051a61-4df7-4398-a699-6f0233f5a1c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 09:19:39.511838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:39.512990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:39.513992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 09:19:39.513992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 09:19:39.513992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 09:19:39.513992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:39.513992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 657/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:19:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:39.519072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83aeec-f57c-43c5-9f4d-97720717ed96 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:39.620583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83aeec-f57c-43c5-9f4d-97720717ed96 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:39.656038 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6162b20-6e87-413d-ae27-9401ef497a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:39.656873 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6162b20-6e87-413d-ae27-9401ef497a73 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:39.657158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6162b20-6e87-413d-ae27-9401ef497a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:39.657435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6162b20-6e87-413d-ae27-9401ef497a73 None admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:39.658657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6162b20-6e87-413d-ae27-9401ef497a73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:39.658964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6162b20-6e87-413d-ae27-9401ef497a73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:39.660363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6162b20-6e87-413d-ae27-9401ef497a73 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:39.671341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 627/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Mon Jun 27 09:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:39.677133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0051a61-4df7-4398-a699-6f0233f5a1c4 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:39.779512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0051a61-4df7-4398-a699-6f0233f5a1c4 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:39.795551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:39.816466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd4a46-464d-4845-9408-f75209e6a43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:39.817266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd4a46-464d-4845-9408-f75209e6a43d None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:39.817548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd4a46-464d-4845-9408-f75209e6a43d None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:39.817822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd4a46-464d-4845-9408-f75209e6a43d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:39.824748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd4a46-464d-4845-9408-f75209e6a43d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:39.826684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd4a46-464d-4845-9408-f75209e6a43d None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application [None req-3abd4a46-464d-4845-9408-f75209e6a43d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 09:19:39.841060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:39.847002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:39.848116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 09:19:39.848116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 09:19:39.848116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 09:19:39.848116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 09:19:39.848116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 09:19:39.848116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 09:19:39.848116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 09:19:39.848116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: ERROR keystone.server.flask.application Jun 27 09:19:39.848116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 658/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Mon Jun 27 09:19:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:39.848917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6162b20-6e87-413d-ae27-9401ef497a73 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:39.905431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:39.959136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6162b20-6e87-413d-ae27-9401ef497a73 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:39.994093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362195f-98cd-44a4-961c-64e8af7b171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:39.994791 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362195f-98cd-44a4-961c-64e8af7b171c None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:39.995095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362195f-98cd-44a4-961c-64e8af7b171c None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:39.995353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362195f-98cd-44a4-961c-64e8af7b171c None admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:39.996175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362195f-98cd-44a4-961c-64e8af7b171c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:39.996487 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362195f-98cd-44a4-961c-64e8af7b171c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:39.998505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362195f-98cd-44a4-961c-64e8af7b171c None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:40.011739 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 628/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Mon Jun 27 09:19:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:40.018624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd4a46-464d-4845-9408-f75209e6a43d None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:40.127260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd4a46-464d-4845-9408-f75209e6a43d None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:40.170355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248a473-76a9-412f-91c1-9004d3954c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:40.171469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248a473-76a9-412f-91c1-9004d3954c34 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:40.171978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248a473-76a9-412f-91c1-9004d3954c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:40.172343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248a473-76a9-412f-91c1-9004d3954c34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:40.173951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248a473-76a9-412f-91c1-9004d3954c34 None admin] RBAC: Adding query filter params (role.id=03b724e03f704f2ab338b64025a3db6d, scope.project.id=84f2268a6c4c456293cdde00cf0026be, user.id=19d4d5fa82dc42e498fb22349921885e) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:40.174503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248a473-76a9-412f-91c1-9004d3954c34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:40.177328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248a473-76a9-412f-91c1-9004d3954c34 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:40.186875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 659/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Mon Jun 27 09:19:40 2022] GET /identity/v3/role_assignments?user.id=19d4d5fa82dc42e498fb22349921885e&scope.project.id=84f2268a6c4c456293cdde00cf0026be&role.id=03b724e03f704f2ab338b64025a3db6d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:19:42.328815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cce01-6674-4471-808a-f7b3a252d283 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:42.329338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cce01-6674-4471-808a-f7b3a252d283 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:42.329426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cce01-6674-4471-808a-f7b3a252d283 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:42.330744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 629/1288] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:42.336127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705150a-7ccf-4395-bfc4-622824ca7e0b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:42.336573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705150a-7ccf-4395-bfc4-622824ca7e0b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:42.336668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705150a-7ccf-4395-bfc4-622824ca7e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:42.353805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5705150a-7ccf-4395-bfc4-622824ca7e0b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:42.354997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5705150a-7ccf-4395-bfc4-622824ca7e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:42.385029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:42.422546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 660/1289] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:19:42.445021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cce01-6674-4471-808a-f7b3a252d283 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:42.548356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cce01-6674-4471-808a-f7b3a252d283 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:42.559097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:42.581380 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c457507-ff47-4f8b-a29a-0ed9275a5abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:42.582243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c457507-ff47-4f8b-a29a-0ed9275a5abb None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:42.582516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c457507-ff47-4f8b-a29a-0ed9275a5abb None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:42.582781 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c457507-ff47-4f8b-a29a-0ed9275a5abb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:42.583422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c457507-ff47-4f8b-a29a-0ed9275a5abb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:42.584871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c457507-ff47-4f8b-a29a-0ed9275a5abb None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-5c457507-ff47-4f8b-a29a-0ed9275a5abb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:42.597287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:42.600563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:42.601631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 630/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:19:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:42.604546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705150a-7ccf-4395-bfc4-622824ca7e0b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:42.705812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705150a-7ccf-4395-bfc4-622824ca7e0b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:42.741919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a41713-1383-4b3d-a854-065b0bcdeac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:42.745728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a41713-1383-4b3d-a854-065b0bcdeac0 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:42.745728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a41713-1383-4b3d-a854-065b0bcdeac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:42.745728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a41713-1383-4b3d-a854-065b0bcdeac0 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:42.745728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a41713-1383-4b3d-a854-065b0bcdeac0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:42.745728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a41713-1383-4b3d-a854-065b0bcdeac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:42.745728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a41713-1383-4b3d-a854-065b0bcdeac0 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:42.750506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 661/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Mon Jun 27 09:19:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:42.756450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c457507-ff47-4f8b-a29a-0ed9275a5abb None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:42.856946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c457507-ff47-4f8b-a29a-0ed9275a5abb None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:42.890065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858274-b93b-422a-9836-4af05dc2851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:42.890715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858274-b93b-422a-9836-4af05dc2851f None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:42.890992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858274-b93b-422a-9836-4af05dc2851f None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:42.891261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858274-b93b-422a-9836-4af05dc2851f None admin] PATH_INFO: `/v3/services` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:42.891998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858274-b93b-422a-9836-4af05dc2851f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:42.892300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858274-b93b-422a-9836-4af05dc2851f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:42.893660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858274-b93b-422a-9836-4af05dc2851f None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:42.898097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 631/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Mon Jun 27 09:19:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:44.963231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b26c5-6693-4bcb-bee3-b279df7559b2 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:44.963797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b26c5-6693-4bcb-bee3-b279df7559b2 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:44.963906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b26c5-6693-4bcb-bee3-b279df7559b2 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:44.965235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 662/1293] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:44.970549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b61de-30db-4ea4-a9e1-780dc8f25641 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:44.970997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b61de-30db-4ea4-a9e1-780dc8f25641 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:44.971101 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b61de-30db-4ea4-a9e1-780dc8f25641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:44.988238 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-374b61de-30db-4ea4-a9e1-780dc8f25641 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:44.990721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-374b61de-30db-4ea4-a9e1-780dc8f25641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:45.019059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:45.056101 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 632/1294] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:19:45.079491 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b26c5-6693-4bcb-bee3-b279df7559b2 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:45.115810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:45.191477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b26c5-6693-4bcb-bee3-b279df7559b2 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:45.227449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055b610-6e51-49a2-95bb-daaf63addcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:45.228100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055b610-6e51-49a2-95bb-daaf63addcea None admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:45.228378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055b610-6e51-49a2-95bb-daaf63addcea None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:45.228657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055b610-6e51-49a2-95bb-daaf63addcea None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:45.229422 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055b610-6e51-49a2-95bb-daaf63addcea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:45.230802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055b610-6e51-49a2-95bb-daaf63addcea None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:45.355271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 663/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Mon Jun 27 09:19:45 2022] POST /identity/v3/services => generated 246 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:19:47.481768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9322e8a-3cad-49f7-ac08-4984efff0c30 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:47.482391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9322e8a-3cad-49f7-ac08-4984efff0c30 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:47.482391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9322e8a-3cad-49f7-ac08-4984efff0c30 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:47.483753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 633/1296] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:47.489204 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5d7a8-1a8f-409e-a509-95c56e73feb8 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:47.489651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5d7a8-1a8f-409e-a509-95c56e73feb8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:47.489747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5d7a8-1a8f-409e-a509-95c56e73feb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:47.507484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-aaf5d7a8-1a8f-409e-a509-95c56e73feb8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:47.508742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-aaf5d7a8-1a8f-409e-a509-95c56e73feb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:47.591404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 664/1297] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:47 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 09:19:47.614793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9322e8a-3cad-49f7-ac08-4984efff0c30 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:47.724974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9322e8a-3cad-49f7-ac08-4984efff0c30 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:47.760048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051dec9-d72a-4c5b-8f1c-fbe13d5a71e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:47.760856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051dec9-d72a-4c5b-8f1c-fbe13d5a71e0 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:47.761141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051dec9-d72a-4c5b-8f1c-fbe13d5a71e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:47.761420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051dec9-d72a-4c5b-8f1c-fbe13d5a71e0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:47.762087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051dec9-d72a-4c5b-8f1c-fbe13d5a71e0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:47.764059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051dec9-d72a-4c5b-8f1c-fbe13d5a71e0 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-5051dec9-d72a-4c5b-8f1c-fbe13d5a71e0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:47.776310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:47.777918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:47.778869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 634/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:19:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:47.783996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5d7a8-1a8f-409e-a509-95c56e73feb8 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:47.803279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:47.888619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5d7a8-1a8f-409e-a509-95c56e73feb8 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:47.924804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a0760-ee77-4916-b8b0-67b3381f3752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:47.925476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a0760-ee77-4916-b8b0-67b3381f3752 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:47.925780 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a0760-ee77-4916-b8b0-67b3381f3752 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:47.926054 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a0760-ee77-4916-b8b0-67b3381f3752 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:47.926779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a0760-ee77-4916-b8b0-67b3381f3752 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:47.927085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a0760-ee77-4916-b8b0-67b3381f3752 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:47.928512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a0760-ee77-4916-b8b0-67b3381f3752 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:47.934053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 665/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Mon Jun 27 09:19:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:47.940639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051dec9-d72a-4c5b-8f1c-fbe13d5a71e0 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:47.971130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:48.039416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051dec9-d72a-4c5b-8f1c-fbe13d5a71e0 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:48.075678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80d26e-7d80-419d-bf3a-72eff44c7134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:48.076343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d26e-7d80-419d-bf3a-72eff44c7134 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:48.076624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d26e-7d80-419d-bf3a-72eff44c7134 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:48.076895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d26e-7d80-419d-bf3a-72eff44c7134 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:48.077694 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80d26e-7d80-419d-bf3a-72eff44c7134 None admin] RBAC: Adding query filter params (interface=public, service_id=3b6ea95921da41a3ad8ccb302966fd89, region_id=RegionOne) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:48.077999 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80d26e-7d80-419d-bf3a-72eff44c7134 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:48.079378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80d26e-7d80-419d-bf3a-72eff44c7134 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:48.085016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 635/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Mon Jun 27 09:19:47 2022] GET /identity/v3/endpoints?service_id=3b6ea95921da41a3ad8ccb302966fd89&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:48.091440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a0760-ee77-4916-b8b0-67b3381f3752 None admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:48.177467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:48.197279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a0760-ee77-4916-b8b0-67b3381f3752 None admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:48.235943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21418f3d-34a3-436e-aba2-386280997a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:48.236645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21418f3d-34a3-436e-aba2-386280997a05 None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:48.236925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21418f3d-34a3-436e-aba2-386280997a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:48.237198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21418f3d-34a3-436e-aba2-386280997a05 None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:48.237878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21418f3d-34a3-436e-aba2-386280997a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:48.239297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21418f3d-34a3-436e-aba2-386280997a05 None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:48.246804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 666/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Mon Jun 27 09:19:48 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:19:50.285665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d16c9-6883-4f6d-88cf-5f99e1aa2877 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:50.286462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d16c9-6883-4f6d-88cf-5f99e1aa2877 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:50.286639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d16c9-6883-4f6d-88cf-5f99e1aa2877 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:50.288376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 636/1302] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:19:50.294640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1a016-df3f-41aa-a20e-c88afbdc78b0 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:50.295241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1a016-df3f-41aa-a20e-c88afbdc78b0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:50.295241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1a016-df3f-41aa-a20e-c88afbdc78b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:50.314650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a9b1a016-df3f-41aa-a20e-c88afbdc78b0 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:19:50.315877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a9b1a016-df3f-41aa-a20e-c88afbdc78b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:19:50.383748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 667/1303] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:19:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 09:19:50.408738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d16c9-6883-4f6d-88cf-5f99e1aa2877 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:50.506929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:50.514480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d16c9-6883-4f6d-88cf-5f99e1aa2877 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:50.549033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab38afa-c712-4d59-b019-431a5dcfdc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:50.549750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab38afa-c712-4d59-b019-431a5dcfdc67 None admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:50.550030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab38afa-c712-4d59-b019-431a5dcfdc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:50.550306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab38afa-c712-4d59-b019-431a5dcfdc67 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:50.550966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab38afa-c712-4d59-b019-431a5dcfdc67 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:50.552372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab38afa-c712-4d59-b019-431a5dcfdc67 None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application [None req-bab38afa-c712-4d59-b019-431a5dcfdc67 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 09:19:50.565908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 09:19:50.567102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application with Lock( Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self._enter() Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application return self.creator() Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application created_value = creator( Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: ERROR keystone.server.flask.application Jun 27 09:19:50.568405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 637/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Mon Jun 27 09:19:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 09:19:50.573150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1a016-df3f-41aa-a20e-c88afbdc78b0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:50.653187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:19:50.676618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1a016-df3f-41aa-a20e-c88afbdc78b0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:50.710466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3976265-3830-44aa-8c07-c07f6f0e8f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:50.711129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3976265-3830-44aa-8c07-c07f6f0e8f0a None admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:50.711415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3976265-3830-44aa-8c07-c07f6f0e8f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:50.711723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3976265-3830-44aa-8c07-c07f6f0e8f0a None admin] PATH_INFO: `/v3/services` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:50.712419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3976265-3830-44aa-8c07-c07f6f0e8f0a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:19:50.712728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3976265-3830-44aa-8c07-c07f6f0e8f0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:50.714156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3976265-3830-44aa-8c07-c07f6f0e8f0a None admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:50.719389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 668/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Mon Jun 27 09:19:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:19:50.725188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab38afa-c712-4d59-b019-431a5dcfdc67 None admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:19:50.826533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab38afa-c712-4d59-b019-431a5dcfdc67 None admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:19:50.859471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738c202-6f93-4cdd-aff0-614de3eda48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:19:50.860157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738c202-6f93-4cdd-aff0-614de3eda48b None admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:19:50.860447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738c202-6f93-4cdd-aff0-614de3eda48b None admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:19:50.860718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738c202-6f93-4cdd-aff0-614de3eda48b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:19:50.861495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738c202-6f93-4cdd-aff0-614de3eda48b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:19:50.862894 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738c202-6f93-4cdd-aff0-614de3eda48b None admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:19:50.972759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 638/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Mon Jun 27 09:19:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:20:00.754210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=30 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:20:00.968514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:43.474884 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5fe00-54e6-422b-8c94-e79f4d75bafc None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:43.475462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5fe00-54e6-422b-8c94-e79f4d75bafc None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:43.475462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5fe00-54e6-422b-8c94-e79f4d75bafc None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:43.482746 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd177ab9-a413-4dcc-a5a3-167f2f9d1cfa None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:43.483266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd177ab9-a413-4dcc-a5a3-167f2f9d1cfa None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:43.483365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd177ab9-a413-4dcc-a5a3-167f2f9d1cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:43.486808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 669/1307] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:27:43 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:27:43.578355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-dd177ab9-a413-4dcc-a5a3-167f2f9d1cfa None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:27:43.579499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-dd177ab9-a413-4dcc-a5a3-167f2f9d1cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:43.711964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 639/1308] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:27:43.751614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaba1c-3e7c-411d-b305-08a76e5249d3 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:43.752182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaba1c-3e7c-411d-b305-08a76e5249d3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:43.752610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaba1c-3e7c-411d-b305-08a76e5249d3 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:43.754500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 670/1309] 10.222.0.64 () {54 vars in 964 bytes} [Mon Jun 27 09:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:27:43.760963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a186f-e735-410d-8f16-79c3fcd3d186 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:43.761288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a186f-e735-410d-8f16-79c3fcd3d186 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:43.761635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a186f-e735-410d-8f16-79c3fcd3d186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:43.859108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-780a186f-e735-410d-8f16-79c3fcd3d186 None None] MFA Rules not processed for user `2b87f03cae6c4ca98a82dd5d981ed224`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:27:43.860313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-780a186f-e735-410d-8f16-79c3fcd3d186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:43.884147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:43.948360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 640/1310] 10.222.0.64 () {58 vars in 1061 bytes} [Mon Jun 27 09:27:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:27:43.959214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaba1c-3e7c-411d-b305-08a76e5249d3 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:27:43.969483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-55eaba1c-3e7c-411d-b305-08a76e5249d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:44.124993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaba1c-3e7c-411d-b305-08a76e5249d3 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:27:44.215293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4fd5e-b985-47dd-9d28-df7a65f1805b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:27:44.216124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4fd5e-b985-47dd-9d28-df7a65f1805b service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:44.216481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4fd5e-b985-47dd-9d28-df7a65f1805b service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:44.216822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4fd5e-b985-47dd-9d28-df7a65f1805b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:44.218711 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1fe4fd5e-b985-47dd-9d28-df7a65f1805b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:44.285580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:44.355073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4fd5e-b985-47dd-9d28-df7a65f1805b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:27:44.356817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe4fd5e-b985-47dd-9d28-df7a65f1805b service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:27:44.379456 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 671/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:27:46.901691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1335e-960e-4de8-a83c-94b77f8d7ae8 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:46.902317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1335e-960e-4de8-a83c-94b77f8d7ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:46.902317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1335e-960e-4de8-a83c-94b77f8d7ae8 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:46.903981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 641/1312] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:27:46.910725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ba2f-2698-4a3e-9cf9-3cf8536fd169 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:46.911617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ba2f-2698-4a3e-9cf9-3cf8536fd169 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:46.911896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ba2f-2698-4a3e-9cf9-3cf8536fd169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:47.008652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-f955ba2f-2698-4a3e-9cf9-3cf8536fd169 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:27:47.009910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f955ba2f-2698-4a3e-9cf9-3cf8536fd169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:47.168888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 672/1313] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:27:47.208375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1335e-960e-4de8-a83c-94b77f8d7ae8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:27:47.210049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-78d1335e-960e-4de8-a83c-94b77f8d7ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:47.334061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1335e-960e-4de8-a83c-94b77f8d7ae8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:27:47.428738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:47.431957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c86180-9f60-47f8-b324-74093e20716d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:27:47.433148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c86180-9f60-47f8-b324-74093e20716d service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:47.433997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c86180-9f60-47f8-b324-74093e20716d service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:47.434290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c86180-9f60-47f8-b324-74093e20716d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:47.436143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-75c86180-9f60-47f8-b324-74093e20716d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:47.566136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c86180-9f60-47f8-b324-74093e20716d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:27:47.567609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c86180-9f60-47f8-b324-74093e20716d service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:27:47.587414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 642/1314] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:27:50.211696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcff3c1-4487-40ed-8976-171e956c807f None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:50.212206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcff3c1-4487-40ed-8976-171e956c807f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:50.212206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcff3c1-4487-40ed-8976-171e956c807f None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:50.213526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 673/1315] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:27:50.219480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fc301-000c-415e-84d5-d60b6d162425 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:50.219933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fc301-000c-415e-84d5-d60b6d162425 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:50.220075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fc301-000c-415e-84d5-d60b6d162425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:50.284148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-291fc301-000c-415e-84d5-d60b6d162425 None None] MFA Rules not processed for user `8390e06ba7334f4d8e1afbf76a9b3591`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:27:50.285376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-291fc301-000c-415e-84d5-d60b6d162425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:50.338263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:50.385087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 643/1316] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:27:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:27:50.425213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcff3c1-4487-40ed-8976-171e956c807f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:27:50.426753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7dcff3c1-4487-40ed-8976-171e956c807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:50.436727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:50.529333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcff3c1-4487-40ed-8976-171e956c807f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:27:50.602152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011cd65-81eb-4144-a906-de803fd7c70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:27:50.602831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011cd65-81eb-4144-a906-de803fd7c70c service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:50.603114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011cd65-81eb-4144-a906-de803fd7c70c service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:50.603389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011cd65-81eb-4144-a906-de803fd7c70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:50.605147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c011cd65-81eb-4144-a906-de803fd7c70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:50.666721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:50.713418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011cd65-81eb-4144-a906-de803fd7c70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:27:50.714879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011cd65-81eb-4144-a906-de803fd7c70c service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:27:50.734312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 674/1317] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:27:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:27:53.586179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dee1e-2c37-4aaa-8360-1d06f4418405 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:53.586692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dee1e-2c37-4aaa-8360-1d06f4418405 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:53.586692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7dee1e-2c37-4aaa-8360-1d06f4418405 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:53.588115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 644/1318] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:27:53.593275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32fe89-f1f2-4dae-a170-acbf226a2b1c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:53.593715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32fe89-f1f2-4dae-a170-acbf226a2b1c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:53.593820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32fe89-f1f2-4dae-a170-acbf226a2b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:53.658057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-de32fe89-f1f2-4dae-a170-acbf226a2b1c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:27:53.659265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-de32fe89-f1f2-4dae-a170-acbf226a2b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:53.754231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 675/1319] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:27:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:27:53.809475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4a0f-6109-440a-a304-be4985cfc775 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:53.809759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4a0f-6109-440a-a304-be4985cfc775 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:53.810034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba4a0f-6109-440a-a304-be4985cfc775 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:53.811431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 645/1320] 10.222.0.64 () {54 vars in 964 bytes} [Mon Jun 27 09:27:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:27:53.817449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcd3f9-6eca-4c67-b0e2-9f3c054aaf79 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:53.817734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcd3f9-6eca-4c67-b0e2-9f3c054aaf79 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:53.818010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcd3f9-6eca-4c67-b0e2-9f3c054aaf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:53.839600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:53.892733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3fdcd3f9-6eca-4c67-b0e2-9f3c054aaf79 None None] MFA Rules not processed for user `2b87f03cae6c4ca98a82dd5d981ed224`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:27:53.893929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3fdcd3f9-6eca-4c67-b0e2-9f3c054aaf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:53.969297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 676/1321] 10.222.0.64 () {58 vars in 1061 bytes} [Mon Jun 27 09:27:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:27:53.978502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4a0f-6109-440a-a304-be4985cfc775 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:27:53.980612 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-57ba4a0f-6109-440a-a304-be4985cfc775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:54.088322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba4a0f-6109-440a-a304-be4985cfc775 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:27:54.160683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e483d8-d641-4539-9efd-232d77b51b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:27:54.161222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e483d8-d641-4539-9efd-232d77b51b3a service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:54.161502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e483d8-d641-4539-9efd-232d77b51b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:54.161775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e483d8-d641-4539-9efd-232d77b51b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:54.163417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e7e483d8-d641-4539-9efd-232d77b51b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:54.171107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:54.274363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e483d8-d641-4539-9efd-232d77b51b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:27:54.275914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e483d8-d641-4539-9efd-232d77b51b3a service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:27:54.295611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 646/1322] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:27:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:27:57.844908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8a96-2283-4332-bf84-99eb6207a470 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:57.845459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8a96-2283-4332-bf84-99eb6207a470 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:57.845459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f8a96-2283-4332-bf84-99eb6207a470 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:57.846974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 677/1323] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:27:57.852016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f5b95-f39f-4dd1-8e9c-76441fb34e24 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:57.852431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f5b95-f39f-4dd1-8e9c-76441fb34e24 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:57.852707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f5b95-f39f-4dd1-8e9c-76441fb34e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:57.919848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d56f5b95-f39f-4dd1-8e9c-76441fb34e24 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:27:57.922054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d56f5b95-f39f-4dd1-8e9c-76441fb34e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:57.983280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:58.024642 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 647/1324] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:27:58.064665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f8a96-2283-4332-bf84-99eb6207a470 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:27:58.066350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-391f8a96-2283-4332-bf84-99eb6207a470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:58.180421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f8a96-2283-4332-bf84-99eb6207a470 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:27:58.193037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:27:58.257527 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ae1fd-dce0-4504-ae34-68b8eca241b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:27:58.258175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ae1fd-dce0-4504-ae34-68b8eca241b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:27:58.258452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ae1fd-dce0-4504-ae34-68b8eca241b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:27:58.258718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ae1fd-dce0-4504-ae34-68b8eca241b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:27:58.260415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9c8ae1fd-dce0-4504-ae34-68b8eca241b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:27:58.372819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ae1fd-dce0-4504-ae34-68b8eca241b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:27:58.374193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ae1fd-dce0-4504-ae34-68b8eca241b4 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:27:58.393074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 678/1325] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:00.485052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a6aa1-14c0-43c3-ae4c-7988e0801950 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:00.485596 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a6aa1-14c0-43c3-ae4c-7988e0801950 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:00.485693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a6aa1-14c0-43c3-ae4c-7988e0801950 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:00.487077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 648/1326] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:00.492446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d37d31-29ff-4616-994a-fa920c01513e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:00.492895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d37d31-29ff-4616-994a-fa920c01513e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:00.492991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d37d31-29ff-4616-994a-fa920c01513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:00.540542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:00.559467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e5d37d31-29ff-4616-994a-fa920c01513e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:00.560703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e5d37d31-29ff-4616-994a-fa920c01513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:00.659607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 679/1327] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:00.695561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a6aa1-14c0-43c3-ae4c-7988e0801950 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:00.697093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1e0a6aa1-14c0-43c3-ae4c-7988e0801950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:00.802929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a6aa1-14c0-43c3-ae4c-7988e0801950 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:00.877712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ffdfd-769f-4c6f-95ff-f3752f682ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:00.878643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ffdfd-769f-4c6f-95ff-f3752f682ebc service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:00.878923 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ffdfd-769f-4c6f-95ff-f3752f682ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:00.879201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ffdfd-769f-4c6f-95ff-f3752f682ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:00.880893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a53ffdfd-769f-4c6f-95ff-f3752f682ebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:00.899342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:00.999841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ffdfd-769f-4c6f-95ff-f3752f682ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:01.001291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ffdfd-769f-4c6f-95ff-f3752f682ebc service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:01.020604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 649/1328] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:03.129464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa870bba-d7ef-493d-bf75-263d0edc9667 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:03.130013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa870bba-d7ef-493d-bf75-263d0edc9667 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:03.130013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa870bba-d7ef-493d-bf75-263d0edc9667 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:03.131380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 680/1329] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:03.137137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c192cda-b551-413e-99aa-da7e5e2d10a9 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:03.137544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c192cda-b551-413e-99aa-da7e5e2d10a9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:03.137728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c192cda-b551-413e-99aa-da7e5e2d10a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:03.203372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1c192cda-b551-413e-99aa-da7e5e2d10a9 None None] MFA Rules not processed for user `8390e06ba7334f4d8e1afbf76a9b3591`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:03.204621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1c192cda-b551-413e-99aa-da7e5e2d10a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:03.270527 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:03.306881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 650/1330] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:03.345934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa870bba-d7ef-493d-bf75-263d0edc9667 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:03.347673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fa870bba-d7ef-493d-bf75-263d0edc9667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:03.463178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa870bba-d7ef-493d-bf75-263d0edc9667 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:03.482815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:03.538742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2206f7e-2896-4a8c-9fb2-ced2b5c8da23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:03.539310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2206f7e-2896-4a8c-9fb2-ced2b5c8da23 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:03.539620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2206f7e-2896-4a8c-9fb2-ced2b5c8da23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:03.539895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2206f7e-2896-4a8c-9fb2-ced2b5c8da23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:03.541529 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d2206f7e-2896-4a8c-9fb2-ced2b5c8da23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:03.656069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2206f7e-2896-4a8c-9fb2-ced2b5c8da23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:03.657439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2206f7e-2896-4a8c-9fb2-ced2b5c8da23 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:03.675575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 681/1331] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:06.232960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f8ab4-e87f-419e-8bba-93990ac07c07 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:06.233499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f8ab4-e87f-419e-8bba-93990ac07c07 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:06.233499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f8ab4-e87f-419e-8bba-93990ac07c07 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:06.234906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 651/1332] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:06.240036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06f76-eb8e-4268-97fd-34728b4242ed None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:06.240466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06f76-eb8e-4268-97fd-34728b4242ed None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:06.240575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06f76-eb8e-4268-97fd-34728b4242ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:06.304275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:06.307111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-6de06f76-eb8e-4268-97fd-34728b4242ed None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:06.308888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6de06f76-eb8e-4268-97fd-34728b4242ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:06.404267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 682/1333] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:06.440535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f8ab4-e87f-419e-8bba-93990ac07c07 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:06.442072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-223f8ab4-e87f-419e-8bba-93990ac07c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:06.544798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f8ab4-e87f-419e-8bba-93990ac07c07 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:06.616864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd9404-82eb-425f-84ce-ce20255066cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:06.617539 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd9404-82eb-425f-84ce-ce20255066cb service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:06.617820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd9404-82eb-425f-84ce-ce20255066cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:06.618094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd9404-82eb-425f-84ce-ce20255066cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:06.619821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-68cd9404-82eb-425f-84ce-ce20255066cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:06.647840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:06.731945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd9404-82eb-425f-84ce-ce20255066cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:06.733386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd9404-82eb-425f-84ce-ce20255066cb service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:06.752343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 652/1334] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:08.839484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287113d-806e-46fe-b870-55a7c9c7fd0d None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:08.840144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287113d-806e-46fe-b870-55a7c9c7fd0d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:08.840144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287113d-806e-46fe-b870-55a7c9c7fd0d None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:08.841410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 683/1335] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:08.846725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646be787-4922-402a-b367-8445e359d87e None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:08.847147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646be787-4922-402a-b367-8445e359d87e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:08.847275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646be787-4922-402a-b367-8445e359d87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:08.910402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-646be787-4922-402a-b367-8445e359d87e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:08.911620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-646be787-4922-402a-b367-8445e359d87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:08.986748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:09.009529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 653/1336] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:28:09.049659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287113d-806e-46fe-b870-55a7c9c7fd0d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:09.051263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c287113d-806e-46fe-b870-55a7c9c7fd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:09.155092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287113d-806e-46fe-b870-55a7c9c7fd0d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:09.187035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:09.232070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15438f36-4303-49d0-b6c3-8cb99ef6dc07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:09.232720 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15438f36-4303-49d0-b6c3-8cb99ef6dc07 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:09.233001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15438f36-4303-49d0-b6c3-8cb99ef6dc07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:09.233273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15438f36-4303-49d0-b6c3-8cb99ef6dc07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:09.234944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-15438f36-4303-49d0-b6c3-8cb99ef6dc07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:09.345556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15438f36-4303-49d0-b6c3-8cb99ef6dc07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:09.346998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15438f36-4303-49d0-b6c3-8cb99ef6dc07 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:09.365553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 684/1337] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:11.980146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bc98f-8120-400c-a29a-d84a812fbf1e None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:11.980674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bc98f-8120-400c-a29a-d84a812fbf1e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:11.980784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bc98f-8120-400c-a29a-d84a812fbf1e None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:11.982178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 654/1338] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:11.987616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd68b6-c402-49e8-a835-727cdfc66c55 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:11.988033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd68b6-c402-49e8-a835-727cdfc66c55 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:11.988177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd68b6-c402-49e8-a835-727cdfc66c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:12.054870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a2bd68b6-c402-49e8-a835-727cdfc66c55 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:12.056055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a2bd68b6-c402-49e8-a835-727cdfc66c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:12.064830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:12.151856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 685/1339] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:12.188954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bc98f-8120-400c-a29a-d84a812fbf1e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:12.190484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0a9bc98f-8120-400c-a29a-d84a812fbf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:12.295354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bc98f-8120-400c-a29a-d84a812fbf1e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:12.370697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab394e8-6305-4032-9126-843c26ed73c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:12.371263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab394e8-6305-4032-9126-843c26ed73c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:12.371563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab394e8-6305-4032-9126-843c26ed73c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:12.371848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab394e8-6305-4032-9126-843c26ed73c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:12.373503 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-aab394e8-6305-4032-9126-843c26ed73c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:12.409179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:12.482523 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab394e8-6305-4032-9126-843c26ed73c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:12.484245 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab394e8-6305-4032-9126-843c26ed73c5 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:12.504746 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 655/1340] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:16.846482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a8ab4-afe9-4995-a9fb-a2e45de70873 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:16.846888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a8ab4-afe9-4995-a9fb-a2e45de70873 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:16.847046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a8ab4-afe9-4995-a9fb-a2e45de70873 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:16.848596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 686/1341] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:16.854313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86cc3a-495e-4dba-a5e3-c4a41953c443 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:16.854799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86cc3a-495e-4dba-a5e3-c4a41953c443 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:16.854910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86cc3a-495e-4dba-a5e3-c4a41953c443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:16.929094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1a86cc3a-495e-4dba-a5e3-c4a41953c443 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:16.930325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1a86cc3a-495e-4dba-a5e3-c4a41953c443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:17.038926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:17.052878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 656/1342] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:17.093216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a8ab4-afe9-4995-a9fb-a2e45de70873 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:17.095439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3c2a8ab4-afe9-4995-a9fb-a2e45de70873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:17.192635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a8ab4-afe9-4995-a9fb-a2e45de70873 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:17.196337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7e03e-6b1c-4101-ac9a-646da1dac928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:17.196985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7e03e-6b1c-4101-ac9a-646da1dac928 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:17.197268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7e03e-6b1c-4101-ac9a-646da1dac928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:17.197635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7e03e-6b1c-4101-ac9a-646da1dac928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:17.208009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7e03e-6b1c-4101-ac9a-646da1dac928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:17.209443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7e03e-6b1c-4101-ac9a-646da1dac928 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:17.219288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 687/1343] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:20.268043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442d2db-4428-4ae8-97cf-3fb557b6110e None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:20.268585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442d2db-4428-4ae8-97cf-3fb557b6110e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:20.268585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442d2db-4428-4ae8-97cf-3fb557b6110e None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:20.270168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 657/1344] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:20.275552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e48b9-ae87-429f-8f2a-2993952ed97c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:20.276015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e48b9-ae87-429f-8f2a-2993952ed97c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:20.276190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e48b9-ae87-429f-8f2a-2993952ed97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:20.305726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-692e48b9-ae87-429f-8f2a-2993952ed97c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:20.306936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-692e48b9-ae87-429f-8f2a-2993952ed97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:20.337794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:20.387669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 688/1345] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:20.426389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442d2db-4428-4ae8-97cf-3fb557b6110e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:20.437543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442d2db-4428-4ae8-97cf-3fb557b6110e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:20.441121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0120192-959d-4065-a44d-eb2d53d87e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:20.441756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0120192-959d-4065-a44d-eb2d53d87e00 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:20.442039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0120192-959d-4065-a44d-eb2d53d87e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:20.442316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0120192-959d-4065-a44d-eb2d53d87e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:20.451796 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0120192-959d-4065-a44d-eb2d53d87e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:20.453224 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0120192-959d-4065-a44d-eb2d53d87e00 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:20.462675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 658/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:24.688182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81414ed-5498-43ef-9158-754edb174f11 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:24.688676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81414ed-5498-43ef-9158-754edb174f11 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:24.690411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81414ed-5498-43ef-9158-754edb174f11 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:24.690411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 689/1347] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:28:24.695785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b9cc0-41dc-41d3-b731-cfbe6c134cbe None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:24.696235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b9cc0-41dc-41d3-b731-cfbe6c134cbe None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:24.696484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b9cc0-41dc-41d3-b731-cfbe6c134cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:24.714533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-093b9cc0-41dc-41d3-b731-cfbe6c134cbe None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:24.715866 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-093b9cc0-41dc-41d3-b731-cfbe6c134cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:24.724001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 659/1348] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:24.762677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81414ed-5498-43ef-9158-754edb174f11 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:24.774209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81414ed-5498-43ef-9158-754edb174f11 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:24.777913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda96a2d-6ede-4aa9-9865-33b0754eb4da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:24.778554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda96a2d-6ede-4aa9-9865-33b0754eb4da service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:24.778830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda96a2d-6ede-4aa9-9865-33b0754eb4da service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:24.779100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda96a2d-6ede-4aa9-9865-33b0754eb4da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:24.789397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda96a2d-6ede-4aa9-9865-33b0754eb4da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:24.790856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda96a2d-6ede-4aa9-9865-33b0754eb4da service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:24.801308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 690/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:29.820263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999352-710a-4824-b640-b711a2473bbf None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:29.821173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999352-710a-4824-b640-b711a2473bbf None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:29.821755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4999352-710a-4824-b640-b711a2473bbf None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:29.825413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 660/1350] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:28:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:29.836453 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313cafa-9bc4-427f-bcc2-b99e6922a703 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:29.844135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313cafa-9bc4-427f-bcc2-b99e6922a703 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:29.844135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313cafa-9bc4-427f-bcc2-b99e6922a703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:29.920671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2313cafa-9bc4-427f-bcc2-b99e6922a703 None None] MFA Rules not processed for user `19d4d5fa82dc42e498fb22349921885e`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:29.921872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2313cafa-9bc4-427f-bcc2-b99e6922a703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:30.003120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 691/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:28:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:34.295186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8d61d-a3e8-4a16-831b-c5a2b1cf9341 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:34.295766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8d61d-a3e8-4a16-831b-c5a2b1cf9341 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:34.295766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8d61d-a3e8-4a16-831b-c5a2b1cf9341 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:34.297162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 661/1352] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:34.302609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c6c29-983e-4742-8172-2a78cc3d05bd None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:34.303055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c6c29-983e-4742-8172-2a78cc3d05bd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:34.303276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c6c29-983e-4742-8172-2a78cc3d05bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:34.324130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-f06c6c29-983e-4742-8172-2a78cc3d05bd None None] MFA Rules not processed for user `19d4d5fa82dc42e498fb22349921885e`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:34.325339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f06c6c29-983e-4742-8172-2a78cc3d05bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:34.334908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 692/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:28:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:34.709730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=13 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:35.813462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ebb1-d175-446d-98bd-45a73c088a28 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:35.814093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ebb1-d175-446d-98bd-45a73c088a28 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:35.814093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ebb1-d175-446d-98bd-45a73c088a28 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:35.815472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 662/1354] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:35.820820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfd72c-59b1-47e3-ad5b-aec836fcd701 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:35.821207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfd72c-59b1-47e3-ad5b-aec836fcd701 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:35.821377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfd72c-59b1-47e3-ad5b-aec836fcd701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:35.840515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c5dfd72c-59b1-47e3-ad5b-aec836fcd701 None None] MFA Rules not processed for user `19d4d5fa82dc42e498fb22349921885e`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:35.841711 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c5dfd72c-59b1-47e3-ad5b-aec836fcd701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:35.850440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 693/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:28:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:39.481311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ebea8-b7a4-4138-8b3c-acbeafbfdadd None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:39.481828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ebea8-b7a4-4138-8b3c-acbeafbfdadd None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:39.481828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ebea8-b7a4-4138-8b3c-acbeafbfdadd None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:39.483337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 663/1356] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:39.489510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35554543-cfc5-43d6-8160-daa4fefe1718 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:39.490015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35554543-cfc5-43d6-8160-daa4fefe1718 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:39.490129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35554543-cfc5-43d6-8160-daa4fefe1718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:39.513808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-35554543-cfc5-43d6-8160-daa4fefe1718 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:39.515515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-35554543-cfc5-43d6-8160-daa4fefe1718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:39.524826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 694/1357] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:39.583797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d902be2-69c2-4a21-b80d-0de55382fb76 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:39.584545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d902be2-69c2-4a21-b80d-0de55382fb76 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:39.584984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d902be2-69c2-4a21-b80d-0de55382fb76 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:39.587100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 664/1358] 10.222.0.64 () {54 vars in 961 bytes} [Mon Jun 27 09:28:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:39.596019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e91b8b-6051-4fc8-a3d0-1c6fa2723786 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:39.598151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e91b8b-6051-4fc8-a3d0-1c6fa2723786 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:39.598151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e91b8b-6051-4fc8-a3d0-1c6fa2723786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:39.643517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:39.652845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-60e91b8b-6051-4fc8-a3d0-1c6fa2723786 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:39.655613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-60e91b8b-6051-4fc8-a3d0-1c6fa2723786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:39.741331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 695/1359] 10.222.0.64 () {58 vars in 1058 bytes} [Mon Jun 27 09:28:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:39.749680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d902be2-69c2-4a21-b80d-0de55382fb76 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:39.764032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d902be2-69c2-4a21-b80d-0de55382fb76 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:39.770669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16e7e7-27e7-49f7-abcd-281d5d21c8b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:39.771692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16e7e7-27e7-49f7-abcd-281d5d21c8b6 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:39.772210 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16e7e7-27e7-49f7-abcd-281d5d21c8b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:39.772721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16e7e7-27e7-49f7-abcd-281d5d21c8b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:39.782239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16e7e7-27e7-49f7-abcd-281d5d21c8b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:39.784149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16e7e7-27e7-49f7-abcd-281d5d21c8b6 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:39.794954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 665/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:42.040329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bfeca-12da-4752-a8bd-dd354f288fcb None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:42.041718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bfeca-12da-4752-a8bd-dd354f288fcb None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:42.041993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bfeca-12da-4752-a8bd-dd354f288fcb None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:42.043612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 696/1361] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:42.049078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814260ee-f0b3-467e-bd44-c7d44b950d8e None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:42.049838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814260ee-f0b3-467e-bd44-c7d44b950d8e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:42.050118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814260ee-f0b3-467e-bd44-c7d44b950d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:42.070116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-814260ee-f0b3-467e-bd44-c7d44b950d8e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:42.071926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-814260ee-f0b3-467e-bd44-c7d44b950d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:42.080383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 666/1362] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:42.125879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bfeca-12da-4752-a8bd-dd354f288fcb None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:42.140532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bfeca-12da-4752-a8bd-dd354f288fcb None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:42.146082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b534e4f-a185-4a57-ab4e-ba770d85fd27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:42.146730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b534e4f-a185-4a57-ab4e-ba770d85fd27 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:42.147008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b534e4f-a185-4a57-ab4e-ba770d85fd27 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:42.147284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b534e4f-a185-4a57-ab4e-ba770d85fd27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:42.156476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b534e4f-a185-4a57-ab4e-ba770d85fd27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:42.158047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b534e4f-a185-4a57-ab4e-ba770d85fd27 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:42.168835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 697/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:44.797105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89cbc5-f708-4700-b7cc-0ea7874a553d None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:44.797646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89cbc5-f708-4700-b7cc-0ea7874a553d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:44.797740 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89cbc5-f708-4700-b7cc-0ea7874a553d None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:44.799068 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 667/1364] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:44.804739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afbb12-b434-48f4-856a-e34ff8e3d756 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:44.805202 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afbb12-b434-48f4-856a-e34ff8e3d756 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:44.805295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afbb12-b434-48f4-856a-e34ff8e3d756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:44.823434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-91afbb12-b434-48f4-856a-e34ff8e3d756 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:44.824684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-91afbb12-b434-48f4-856a-e34ff8e3d756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:44.832737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 698/1365] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:44.880564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba1e57-cd63-4705-a18c-8b4e864f220c None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:44.880934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba1e57-cd63-4705-a18c-8b4e864f220c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:44.881363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba1e57-cd63-4705-a18c-8b4e864f220c None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:44.882949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 668/1366] 10.222.0.64 () {54 vars in 961 bytes} [Mon Jun 27 09:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:44.888283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b95bc-5608-4e29-8440-9368f68c2d63 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:44.888577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b95bc-5608-4e29-8440-9368f68c2d63 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:44.888854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b95bc-5608-4e29-8440-9368f68c2d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:44.907772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-105b95bc-5608-4e29-8440-9368f68c2d63 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:44.909212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-105b95bc-5608-4e29-8440-9368f68c2d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:44.919406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 699/1367] 10.222.0.64 () {58 vars in 1058 bytes} [Mon Jun 27 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:44.927466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba1e57-cd63-4705-a18c-8b4e864f220c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:44.942198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba1e57-cd63-4705-a18c-8b4e864f220c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:44.948174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5ea64-999f-4e4b-9c6a-7bd0d73850d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:44.948812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5ea64-999f-4e4b-9c6a-7bd0d73850d0 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:44.949097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5ea64-999f-4e4b-9c6a-7bd0d73850d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:44.949376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5ea64-999f-4e4b-9c6a-7bd0d73850d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:44.957813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5ea64-999f-4e4b-9c6a-7bd0d73850d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:44.959564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5ea64-999f-4e4b-9c6a-7bd0d73850d0 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:44.970366 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 669/1368] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:47.299819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2aad4-ead8-4266-bd5c-e402d689afc9 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:47.300708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2aad4-ead8-4266-bd5c-e402d689afc9 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:47.300991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2aad4-ead8-4266-bd5c-e402d689afc9 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:47.302381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 700/1369] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:47.308534 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3fb71-5861-42d4-9617-2244c0e70a65 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:47.309124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3fb71-5861-42d4-9617-2244c0e70a65 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:47.309124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3fb71-5861-42d4-9617-2244c0e70a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:47.328311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4bf3fb71-5861-42d4-9617-2244c0e70a65 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:47.329653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4bf3fb71-5861-42d4-9617-2244c0e70a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:47.338413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 670/1370] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:28:47.382907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2aad4-ead8-4266-bd5c-e402d689afc9 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:47.398591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2aad4-ead8-4266-bd5c-e402d689afc9 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:47.404951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e820710-6b99-405e-a1b9-5c723bd633e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:47.405609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e820710-6b99-405e-a1b9-5c723bd633e1 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:47.405893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e820710-6b99-405e-a1b9-5c723bd633e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:47.406173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e820710-6b99-405e-a1b9-5c723bd633e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:47.415667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e820710-6b99-405e-a1b9-5c723bd633e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:47.417284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e820710-6b99-405e-a1b9-5c723bd633e1 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:47.428477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 701/1371] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:47.740955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc37875-6fca-4882-9e3f-233c795e401a None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:47.741488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc37875-6fca-4882-9e3f-233c795e401a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:47.741769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc37875-6fca-4882-9e3f-233c795e401a None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:47.743346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 671/1372] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:47.751817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be505b73-f1d1-49cc-bd80-6c9ad7e368c7 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:47.751817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be505b73-f1d1-49cc-bd80-6c9ad7e368c7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:47.751817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be505b73-f1d1-49cc-bd80-6c9ad7e368c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:47.770244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-be505b73-f1d1-49cc-bd80-6c9ad7e368c7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:47.771481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-be505b73-f1d1-49cc-bd80-6c9ad7e368c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:47.779967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 702/1373] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:48.096022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3666088-ecbd-467e-b0ea-8a2fa1106b0b None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:48.096505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3666088-ecbd-467e-b0ea-8a2fa1106b0b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:48.096783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3666088-ecbd-467e-b0ea-8a2fa1106b0b None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:48.098149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 672/1374] 10.222.0.64 () {54 vars in 963 bytes} [Mon Jun 27 09:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:48.103644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d1c08-ee1b-46cd-9f8d-7959a4760e22 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:48.104131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d1c08-ee1b-46cd-9f8d-7959a4760e22 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:48.104565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d1c08-ee1b-46cd-9f8d-7959a4760e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:48.160843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7d7d1c08-ee1b-46cd-9f8d-7959a4760e22 None None] MFA Rules not processed for user `8c8abb981d1b4c82a43b5a3372f98398`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:48.162313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7d7d1c08-ee1b-46cd-9f8d-7959a4760e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:48.229461 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 703/1375] 10.222.0.64 () {58 vars in 1060 bytes} [Mon Jun 27 09:28:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:48.237394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3666088-ecbd-467e-b0ea-8a2fa1106b0b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:48.249662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3666088-ecbd-467e-b0ea-8a2fa1106b0b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:48.253462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a71f-a2e4-4cbc-9f8e-7b51e488a2d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:48.254275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a71f-a2e4-4cbc-9f8e-7b51e488a2d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:48.254728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a71f-a2e4-4cbc-9f8e-7b51e488a2d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:48.255145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a71f-a2e4-4cbc-9f8e-7b51e488a2d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:48.265416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a71f-a2e4-4cbc-9f8e-7b51e488a2d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:48.267105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a71f-a2e4-4cbc-9f8e-7b51e488a2d1 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:48.278812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 673/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:28:49.767474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a2fdd-3400-453e-bd31-286c639c6401 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:49.767960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a2fdd-3400-453e-bd31-286c639c6401 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:49.768150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a2fdd-3400-453e-bd31-286c639c6401 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:49.769571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 704/1377] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:49.775121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769683b-1459-4ff4-84c0-c5fdf667078d None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:49.775574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769683b-1459-4ff4-84c0-c5fdf667078d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:49.775574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769683b-1459-4ff4-84c0-c5fdf667078d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:49.793975 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-2769683b-1459-4ff4-84c0-c5fdf667078d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:49.795168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2769683b-1459-4ff4-84c0-c5fdf667078d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:49.803876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 674/1378] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:49.848427 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a2fdd-3400-453e-bd31-286c639c6401 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:49.863997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a2fdd-3400-453e-bd31-286c639c6401 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:49.869444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913535ba-11fb-49c3-8f74-44aca52b1247 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:49.870081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913535ba-11fb-49c3-8f74-44aca52b1247 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:49.870380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913535ba-11fb-49c3-8f74-44aca52b1247 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:49.870660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913535ba-11fb-49c3-8f74-44aca52b1247 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:49.879814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913535ba-11fb-49c3-8f74-44aca52b1247 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:49.881563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913535ba-11fb-49c3-8f74-44aca52b1247 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:49.885780 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:28:49.892084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 705/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:52.198258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45647a67-0210-41d5-8769-fca6a0d779a6 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:52.198780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45647a67-0210-41d5-8769-fca6a0d779a6 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:52.198780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45647a67-0210-41d5-8769-fca6a0d779a6 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:52.200120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 675/1380] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:52.205559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92306c32-ef85-478f-abfe-2c49299e7af9 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:52.205969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92306c32-ef85-478f-abfe-2c49299e7af9 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:52.206124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92306c32-ef85-478f-abfe-2c49299e7af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:52.224632 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-92306c32-ef85-478f-abfe-2c49299e7af9 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:52.225827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-92306c32-ef85-478f-abfe-2c49299e7af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:52.234415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 706/1381] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:52.278514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45647a67-0210-41d5-8769-fca6a0d779a6 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:52.300207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45647a67-0210-41d5-8769-fca6a0d779a6 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:52.307150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe29f6-07ee-4dcd-8f77-130937a3ce7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:52.307938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe29f6-07ee-4dcd-8f77-130937a3ce7d service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:52.308288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe29f6-07ee-4dcd-8f77-130937a3ce7d service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:52.308638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe29f6-07ee-4dcd-8f77-130937a3ce7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:52.317862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe29f6-07ee-4dcd-8f77-130937a3ce7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:52.319646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe29f6-07ee-4dcd-8f77-130937a3ce7d service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:52.329905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 676/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:54.590083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fc6bb-01a6-42bb-a04b-3fb29aaaa576 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:54.590594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fc6bb-01a6-42bb-a04b-3fb29aaaa576 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:54.590594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fc6bb-01a6-42bb-a04b-3fb29aaaa576 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:54.592118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 707/1383] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:54.597527 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e73f01-b0a4-456c-9a9a-52a24a4c1b94 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:54.597941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e73f01-b0a4-456c-9a9a-52a24a4c1b94 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:54.598093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e73f01-b0a4-456c-9a9a-52a24a4c1b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:54.615996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f9e73f01-b0a4-456c-9a9a-52a24a4c1b94 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:54.617199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f9e73f01-b0a4-456c-9a9a-52a24a4c1b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:54.625834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 677/1384] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:54.668265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fc6bb-01a6-42bb-a04b-3fb29aaaa576 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:54.684267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fc6bb-01a6-42bb-a04b-3fb29aaaa576 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:54.691172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9231a1-047b-4cc5-92fb-d25ee89b46ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:54.691803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9231a1-047b-4cc5-92fb-d25ee89b46ea service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:54.692307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9231a1-047b-4cc5-92fb-d25ee89b46ea service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:54.699610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9231a1-047b-4cc5-92fb-d25ee89b46ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:54.703050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9231a1-047b-4cc5-92fb-d25ee89b46ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:54.704669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9231a1-047b-4cc5-92fb-d25ee89b46ea service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:54.716050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 708/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:56.802673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb89605-7d20-4e7a-a025-36d3c5298edb None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:56.803258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb89605-7d20-4e7a-a025-36d3c5298edb None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:56.803412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb89605-7d20-4e7a-a025-36d3c5298edb None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:56.805638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 678/1386] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:56.811032 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8513279-28b0-44a7-bd5c-b87d8d60e890 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:56.811679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8513279-28b0-44a7-bd5c-b87d8d60e890 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:56.811679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8513279-28b0-44a7-bd5c-b87d8d60e890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:56.829528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e8513279-28b0-44a7-bd5c-b87d8d60e890 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:56.830767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e8513279-28b0-44a7-bd5c-b87d8d60e890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:56.839565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 709/1387] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:56.885049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb89605-7d20-4e7a-a025-36d3c5298edb None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:56.905248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb89605-7d20-4e7a-a025-36d3c5298edb None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:56.912960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56fcf0-eb05-4ebc-ae34-d045800fc035 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:56.914019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56fcf0-eb05-4ebc-ae34-d045800fc035 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:56.914486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56fcf0-eb05-4ebc-ae34-d045800fc035 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:56.914854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56fcf0-eb05-4ebc-ae34-d045800fc035 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:56.928071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56fcf0-eb05-4ebc-ae34-d045800fc035 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:56.930224 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56fcf0-eb05-4ebc-ae34-d045800fc035 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:56.947359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 679/1388] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:28:59.337936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9321fe-fa16-43dd-8990-b86e742fb485 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:59.338675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9321fe-fa16-43dd-8990-b86e742fb485 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:59.338675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9321fe-fa16-43dd-8990-b86e742fb485 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:59.339703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 710/1389] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:28:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:28:59.345366 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afedd85-4793-4de8-b498-99e2459fa131 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:59.345972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afedd85-4793-4de8-b498-99e2459fa131 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:59.345972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afedd85-4793-4de8-b498-99e2459fa131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:59.365070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8afedd85-4793-4de8-b498-99e2459fa131 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:28:59.366358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8afedd85-4793-4de8-b498-99e2459fa131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:28:59.374834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 680/1390] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:28:59.424700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9321fe-fa16-43dd-8990-b86e742fb485 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:28:59.440559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9321fe-fa16-43dd-8990-b86e742fb485 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:28:59.446258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c44719-e6b7-45e6-ac36-fa13eed38556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:28:59.446799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44719-e6b7-45e6-ac36-fa13eed38556 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:28:59.447082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44719-e6b7-45e6-ac36-fa13eed38556 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:28:59.447368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44719-e6b7-45e6-ac36-fa13eed38556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:28:59.455845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c44719-e6b7-45e6-ac36-fa13eed38556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:28:59.457433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c44719-e6b7-45e6-ac36-fa13eed38556 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:28:59.467016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 711/1391] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:01.812402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82648a6c-1533-4e37-b1ab-a2c9b017d045 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:01.813147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82648a6c-1533-4e37-b1ab-a2c9b017d045 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:01.813674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82648a6c-1533-4e37-b1ab-a2c9b017d045 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:01.817194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 681/1392] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:29:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:01.822852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95868a-7e32-413c-b148-fe6f4ffeafe7 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:01.823282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95868a-7e32-413c-b148-fe6f4ffeafe7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:01.823414 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95868a-7e32-413c-b148-fe6f4ffeafe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:01.842546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5c95868a-7e32-413c-b148-fe6f4ffeafe7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:01.843796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5c95868a-7e32-413c-b148-fe6f4ffeafe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:01.851897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 712/1393] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:01.896633 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82648a6c-1533-4e37-b1ab-a2c9b017d045 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:01.916649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82648a6c-1533-4e37-b1ab-a2c9b017d045 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:01.923571 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152d28b-f5e1-4ee3-9348-7aa188b0f9ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:01.924233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d28b-f5e1-4ee3-9348-7aa188b0f9ef service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:01.924513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d28b-f5e1-4ee3-9348-7aa188b0f9ef service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:01.924785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d28b-f5e1-4ee3-9348-7aa188b0f9ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:01.934566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152d28b-f5e1-4ee3-9348-7aa188b0f9ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:01.936402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152d28b-f5e1-4ee3-9348-7aa188b0f9ef service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:01.946873 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 682/1394] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:04.155785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaad9d-8439-49bc-8188-fcaeecba0254 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:04.156402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaad9d-8439-49bc-8188-fcaeecba0254 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:04.156402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaad9d-8439-49bc-8188-fcaeecba0254 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:04.157683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 713/1395] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:04.163475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdaec6-248c-4fb5-8ca7-dbcb5f5776b1 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:04.164044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdaec6-248c-4fb5-8ca7-dbcb5f5776b1 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:04.164159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdaec6-248c-4fb5-8ca7-dbcb5f5776b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:04.182470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-97fdaec6-248c-4fb5-8ca7-dbcb5f5776b1 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:04.183757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-97fdaec6-248c-4fb5-8ca7-dbcb5f5776b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:04.193671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 683/1396] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:04.221906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d5777-2f98-49ea-a19c-796472ea37ab None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:04.221906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d5777-2f98-49ea-a19c-796472ea37ab None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:04.221906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d5777-2f98-49ea-a19c-796472ea37ab None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:04.223237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 714/1397] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:04.229364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33601471-950e-4429-b28d-df2986a750db None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:04.230160 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33601471-950e-4429-b28d-df2986a750db None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:04.230326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33601471-950e-4429-b28d-df2986a750db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:04.252826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaea38-9dfa-4a3f-8e2d-2302d9ff9312 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:04.253110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaea38-9dfa-4a3f-8e2d-2302d9ff9312 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:04.253390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaea38-9dfa-4a3f-8e2d-2302d9ff9312 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:04.253808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-33601471-950e-4429-b28d-df2986a750db None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:04.254798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 715/1398] 10.222.0.64 () {54 vars in 962 bytes} [Mon Jun 27 09:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:29:04.255118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-33601471-950e-4429-b28d-df2986a750db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:04.261616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c5b52-6c6a-4572-8b94-5e24e7b711fc None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:04.261904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c5b52-6c6a-4572-8b94-5e24e7b711fc None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:04.262175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c5b52-6c6a-4572-8b94-5e24e7b711fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:04.279562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 684/1399] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:04.312058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33601471-950e-4429-b28d-df2986a750db None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:04.322417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-047c5b52-6c6a-4572-8b94-5e24e7b711fc None None] MFA Rules not processed for user `e33d1473077247cc97dcfd99b3da5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:04.323669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-047c5b52-6c6a-4572-8b94-5e24e7b711fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:04.328242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33601471-950e-4429-b28d-df2986a750db None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:04.336037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4300cda-7c85-4649-baf9-4a8069783464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:04.336716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4300cda-7c85-4649-baf9-4a8069783464 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:04.336997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4300cda-7c85-4649-baf9-4a8069783464 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:04.337282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4300cda-7c85-4649-baf9-4a8069783464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:04.346010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4300cda-7c85-4649-baf9-4a8069783464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:04.347641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4300cda-7c85-4649-baf9-4a8069783464 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:04.357140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 685/1400] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:29:04.387387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 716/1401] 10.222.0.64 () {58 vars in 1059 bytes} [Mon Jun 27 09:29:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:29:04.395373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4300cda-7c85-4649-baf9-4a8069783464 service nova] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:04.404939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4300cda-7c85-4649-baf9-4a8069783464 service nova] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:04.409077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b2cee-b1a8-4bf6-ab3c-04e89729b3df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e33d1473077247cc97dcfd99b3da5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:04.409678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b2cee-b1a8-4bf6-ab3c-04e89729b3df service glance] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:04.409989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b2cee-b1a8-4bf6-ab3c-04e89729b3df service glance] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:04.410268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b2cee-b1a8-4bf6-ab3c-04e89729b3df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:04.420308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b2cee-b1a8-4bf6-ab3c-04e89729b3df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:04.421776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b2cee-b1a8-4bf6-ab3c-04e89729b3df service glance] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:04.432864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 686/1402] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:05.350725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585afcd5-747b-4703-a042-fb751785887c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:05.351169 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585afcd5-747b-4703-a042-fb751785887c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:05.352332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585afcd5-747b-4703-a042-fb751785887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:05.398376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-585afcd5-747b-4703-a042-fb751785887c None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:05.400259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-585afcd5-747b-4703-a042-fb751785887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:05.463060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:29:05.466750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 717/1403] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:29:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:05.513320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec8857-681d-432f-9909-0e87b1de4189 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:05.513839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec8857-681d-432f-9909-0e87b1de4189 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:05.513936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec8857-681d-432f-9909-0e87b1de4189 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:05.515284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 687/1404] 10.222.0.64 () {54 vars in 942 bytes} [Mon Jun 27 09:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:05.522862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f6090-30ca-458b-8deb-b2dc977fb24b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:05.523152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f6090-30ca-458b-8deb-b2dc977fb24b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:05.523448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f6090-30ca-458b-8deb-b2dc977fb24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:05.574874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5f9f6090-30ca-458b-8deb-b2dc977fb24b None None] MFA Rules not processed for user `70b499a32516486d9a2bed34f7e66b76`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:05.576097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5f9f6090-30ca-458b-8deb-b2dc977fb24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:05.657851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 718/1405] 10.222.0.64 () {58 vars in 1039 bytes} [Mon Jun 27 09:29:05 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:05.668686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec8857-681d-432f-9909-0e87b1de4189 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:05.676695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=70 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:29:05.683870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec8857-681d-432f-9909-0e87b1de4189 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:05.689158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b867fe-b894-4435-a428-302cb1693f0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:05.689842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b867fe-b894-4435-a428-302cb1693f0b service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:05.690122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b867fe-b894-4435-a428-302cb1693f0b service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:05.690436 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b867fe-b894-4435-a428-302cb1693f0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:05.700187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b867fe-b894-4435-a428-302cb1693f0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:05.701837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b867fe-b894-4435-a428-302cb1693f0b service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:05.711046 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 688/1406] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:29:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:29:06.652261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e94ed2-635a-45ac-b18d-b25f5838359e None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:06.653319 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e94ed2-635a-45ac-b18d-b25f5838359e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:06.653763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e94ed2-635a-45ac-b18d-b25f5838359e None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:06.655192 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 719/1407] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:06.661229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ba11-1bf7-436c-8549-1b174919356a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:06.661719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ba11-1bf7-436c-8549-1b174919356a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:06.661819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ba11-1bf7-436c-8549-1b174919356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:06.681654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7604ba11-1bf7-436c-8549-1b174919356a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:06.683151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7604ba11-1bf7-436c-8549-1b174919356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:06.691982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 689/1408] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:29:06.739680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e94ed2-635a-45ac-b18d-b25f5838359e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:06.754337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e94ed2-635a-45ac-b18d-b25f5838359e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:06.760779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed460cb8-8eca-42ac-9042-e4362e5f8b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:06.761555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed460cb8-8eca-42ac-9042-e4362e5f8b64 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:06.761911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed460cb8-8eca-42ac-9042-e4362e5f8b64 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:06.762258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed460cb8-8eca-42ac-9042-e4362e5f8b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:06.771699 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed460cb8-8eca-42ac-9042-e4362e5f8b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:06.773383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed460cb8-8eca-42ac-9042-e4362e5f8b64 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:06.787515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 720/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:16.675211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=8 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:29:16.782873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=50 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:29:29.959546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7d4c1-c6c9-40f8-96dc-f99101a3c2fa None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:29.960108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7d4c1-c6c9-40f8-96dc-f99101a3c2fa None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:29.960108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7d4c1-c6c9-40f8-96dc-f99101a3c2fa None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:29.962114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 690/1410] 10.222.0.64 () {54 vars in 918 bytes} [Mon Jun 27 09:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:29.967731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4384de-4907-4ad1-b042-03c2e58cb590 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:29.968191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4384de-4907-4ad1-b042-03c2e58cb590 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:29.968363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4384de-4907-4ad1-b042-03c2e58cb590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:29.986308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-9d4384de-4907-4ad1-b042-03c2e58cb590 None None] MFA Rules not processed for user `19d4d5fa82dc42e498fb22349921885e`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:29.987518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9d4384de-4907-4ad1-b042-03c2e58cb590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:29.996126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 721/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Mon Jun 27 09:29:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:30.011637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ec0d6-2f3d-4a25-8a45-793d5765e75a None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:30.011957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ec0d6-2f3d-4a25-8a45-793d5765e75a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:30.012239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ec0d6-2f3d-4a25-8a45-793d5765e75a None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:30.013617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 691/1412] 10.222.0.64 () {54 vars in 918 bytes} [Mon Jun 27 09:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:30.019025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a27d6-8bd6-4dc7-a8bf-85b28c3c932d None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:30.019314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a27d6-8bd6-4dc7-a8bf-85b28c3c932d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:30.019615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a27d6-8bd6-4dc7-a8bf-85b28c3c932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:30.039071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-bd1a27d6-8bd6-4dc7-a8bf-85b28c3c932d None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:30.040307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-bd1a27d6-8bd6-4dc7-a8bf-85b28c3c932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:30.049750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 722/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Mon Jun 27 09:29:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:29:32.562616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a77e5c-eb93-4872-b593-70eebd7ab526 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:32.563168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a77e5c-eb93-4872-b593-70eebd7ab526 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:32.563168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a77e5c-eb93-4872-b593-70eebd7ab526 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:32.564469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 692/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:32.569856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d0a5-8cf0-4c58-903e-6f981cc80e3c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:32.570837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d0a5-8cf0-4c58-903e-6f981cc80e3c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:32.571040 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d0a5-8cf0-4c58-903e-6f981cc80e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:32.589421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-6a70d0a5-8cf0-4c58-903e-6f981cc80e3c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:32.590768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6a70d0a5-8cf0-4c58-903e-6f981cc80e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:32.598793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 723/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:32.641966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a77e5c-eb93-4872-b593-70eebd7ab526 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:32.656437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a77e5c-eb93-4872-b593-70eebd7ab526 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:32.662387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5fc96-17d8-43da-b3ee-00cac0b0ef73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:32.663021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5fc96-17d8-43da-b3ee-00cac0b0ef73 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:32.663303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5fc96-17d8-43da-b3ee-00cac0b0ef73 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:32.663628 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5fc96-17d8-43da-b3ee-00cac0b0ef73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:32.671959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5fc96-17d8-43da-b3ee-00cac0b0ef73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:32.673581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5fc96-17d8-43da-b3ee-00cac0b0ef73 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:32.683628 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 693/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:33.594617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cb3a6-40ee-4b6f-ba6e-28579f24a96b None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:33.595178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cb3a6-40ee-4b6f-ba6e-28579f24a96b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:33.595178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cb3a6-40ee-4b6f-ba6e-28579f24a96b None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:33.596494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 724/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:33.602308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bdc4e-e0cc-4e11-a806-0a76a731d4be None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:33.602720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bdc4e-e0cc-4e11-a806-0a76a731d4be None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:33.602875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bdc4e-e0cc-4e11-a806-0a76a731d4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:33.620806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-c07bdc4e-e0cc-4e11-a806-0a76a731d4be None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:33.622035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c07bdc4e-e0cc-4e11-a806-0a76a731d4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:33.630003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 694/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:33.780617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cb3a6-40ee-4b6f-ba6e-28579f24a96b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:33.795897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cb3a6-40ee-4b6f-ba6e-28579f24a96b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:33.801673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534efc7-e845-4984-a8f6-b570fb43c423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:33.802323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534efc7-e845-4984-a8f6-b570fb43c423 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:33.802621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534efc7-e845-4984-a8f6-b570fb43c423 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:33.802905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534efc7-e845-4984-a8f6-b570fb43c423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:33.812110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534efc7-e845-4984-a8f6-b570fb43c423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:33.813722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534efc7-e845-4984-a8f6-b570fb43c423 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:33.823381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 725/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:34.711663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8906ad-aca7-46ee-9d9b-a0d7832c36f8 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:34.712173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8906ad-aca7-46ee-9d9b-a0d7832c36f8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:34.712173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8906ad-aca7-46ee-9d9b-a0d7832c36f8 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:34.713543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 695/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:34.718783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4db650-8daa-4034-9cae-0d7c2bb40574 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:34.719239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4db650-8daa-4034-9cae-0d7c2bb40574 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:34.719369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4db650-8daa-4034-9cae-0d7c2bb40574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:34.737485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-9c4db650-8daa-4034-9cae-0d7c2bb40574 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:34.738685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9c4db650-8daa-4034-9cae-0d7c2bb40574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:34.747119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 726/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:34.897454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8906ad-aca7-46ee-9d9b-a0d7832c36f8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:34.912470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8906ad-aca7-46ee-9d9b-a0d7832c36f8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:34.918689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225df58-0d8e-4385-b762-441852d339ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:34.919327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225df58-0d8e-4385-b762-441852d339ab service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:34.919670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225df58-0d8e-4385-b762-441852d339ab service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:34.919952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225df58-0d8e-4385-b762-441852d339ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:34.928834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225df58-0d8e-4385-b762-441852d339ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:34.930440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225df58-0d8e-4385-b762-441852d339ab service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:34.940435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 696/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:35.797821 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1d9bd-67f3-4498-9a56-3f23c6e9baf0 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:35.798435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1d9bd-67f3-4498-9a56-3f23c6e9baf0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:35.798435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1d9bd-67f3-4498-9a56-3f23c6e9baf0 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:35.799752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 727/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:35.805184 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db4eab-d1b4-4bda-bd80-16bbb55b307d None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:35.805590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db4eab-d1b4-4bda-bd80-16bbb55b307d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:35.805765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db4eab-d1b4-4bda-bd80-16bbb55b307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:35.824407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-75db4eab-d1b4-4bda-bd80-16bbb55b307d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:35.825710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-75db4eab-d1b4-4bda-bd80-16bbb55b307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:35.834014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 697/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:29:35.986229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1d9bd-67f3-4498-9a56-3f23c6e9baf0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:36.001226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1d9bd-67f3-4498-9a56-3f23c6e9baf0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:36.006696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6cf98-b770-4e6d-8278-88b63f0d8b50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:36.007342 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6cf98-b770-4e6d-8278-88b63f0d8b50 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:36.007658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6cf98-b770-4e6d-8278-88b63f0d8b50 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:36.007940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6cf98-b770-4e6d-8278-88b63f0d8b50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:36.016513 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6cf98-b770-4e6d-8278-88b63f0d8b50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:36.018109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6cf98-b770-4e6d-8278-88b63f0d8b50 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:36.028051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 728/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:36.894580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d4db5-8b76-44ad-80bd-39ca1ef282d3 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:36.895156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d4db5-8b76-44ad-80bd-39ca1ef282d3 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:36.895156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d4db5-8b76-44ad-80bd-39ca1ef282d3 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:36.896563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 698/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:36.902107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179333bd-ae42-4933-911d-c19f537340d0 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:36.902568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179333bd-ae42-4933-911d-c19f537340d0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:36.902673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179333bd-ae42-4933-911d-c19f537340d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:36.921020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-179333bd-ae42-4933-911d-c19f537340d0 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:36.926016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-179333bd-ae42-4933-911d-c19f537340d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:36.930148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 729/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:37.080957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d4db5-8b76-44ad-80bd-39ca1ef282d3 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:37.095913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d4db5-8b76-44ad-80bd-39ca1ef282d3 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:37.101994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635befa-3325-461d-9f5c-8af9dca4e7b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:37.102630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635befa-3325-461d-9f5c-8af9dca4e7b5 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:37.102908 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635befa-3325-461d-9f5c-8af9dca4e7b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:37.103179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635befa-3325-461d-9f5c-8af9dca4e7b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:37.111386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635befa-3325-461d-9f5c-8af9dca4e7b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:37.113116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635befa-3325-461d-9f5c-8af9dca4e7b5 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:37.122703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 699/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:38.040176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd1554-a8ba-4938-b111-1c3b3a353a69 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:38.040765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd1554-a8ba-4938-b111-1c3b3a353a69 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:38.040765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd1554-a8ba-4938-b111-1c3b3a353a69 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:38.042127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 730/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:38.047074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9dcb-15a7-4c7a-aa57-125d0e7a5a14 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:38.047483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9dcb-15a7-4c7a-aa57-125d0e7a5a14 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:38.047677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9dcb-15a7-4c7a-aa57-125d0e7a5a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:38.065935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ce2f9dcb-15a7-4c7a-aa57-125d0e7a5a14 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:38.067149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ce2f9dcb-15a7-4c7a-aa57-125d0e7a5a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:38.075040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 700/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:38.229512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd1554-a8ba-4938-b111-1c3b3a353a69 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:38.243849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd1554-a8ba-4938-b111-1c3b3a353a69 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:38.249176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e4b46-9de7-4f42-bb97-d4fa593e7496 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:38.250230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e4b46-9de7-4f42-bb97-d4fa593e7496 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:38.250508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e4b46-9de7-4f42-bb97-d4fa593e7496 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:38.251010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e4b46-9de7-4f42-bb97-d4fa593e7496 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:38.260417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e4b46-9de7-4f42-bb97-d4fa593e7496 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:38.262009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e4b46-9de7-4f42-bb97-d4fa593e7496 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:38.272796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 731/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:39.218291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fa68a-89f3-47ea-96e4-20e0eaed6736 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:39.218862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fa68a-89f3-47ea-96e4-20e0eaed6736 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:39.218862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fa68a-89f3-47ea-96e4-20e0eaed6736 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:39.220283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 701/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:39.225262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a0a9c-bf23-40a5-be0e-992abe46af45 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:39.225703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a0a9c-bf23-40a5-be0e-992abe46af45 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:39.225887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a0a9c-bf23-40a5-be0e-992abe46af45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:39.245649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-6a8a0a9c-bf23-40a5-be0e-992abe46af45 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:39.246836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6a8a0a9c-bf23-40a5-be0e-992abe46af45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:39.254564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 732/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:39.405923 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fa68a-89f3-47ea-96e4-20e0eaed6736 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:39.420953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fa68a-89f3-47ea-96e4-20e0eaed6736 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:39.426311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32850e-3649-4c24-adc6-4bd0d0a9f46e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:39.426950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32850e-3649-4c24-adc6-4bd0d0a9f46e service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:39.427227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32850e-3649-4c24-adc6-4bd0d0a9f46e service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:39.427506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32850e-3649-4c24-adc6-4bd0d0a9f46e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:39.435518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32850e-3649-4c24-adc6-4bd0d0a9f46e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:39.437123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32850e-3649-4c24-adc6-4bd0d0a9f46e service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:39.446427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 702/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:40.309450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9dc5-16ba-4686-aed7-6c3a099d2e3b None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:40.309984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9dc5-16ba-4686-aed7-6c3a099d2e3b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:40.309984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9dc5-16ba-4686-aed7-6c3a099d2e3b None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:40.311373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 733/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:40.316463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37910d-96c2-4f5a-85c1-418b17de4367 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:40.316898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37910d-96c2-4f5a-85c1-418b17de4367 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:40.317030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37910d-96c2-4f5a-85c1-418b17de4367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:40.335244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-dc37910d-96c2-4f5a-85c1-418b17de4367 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:40.339303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-dc37910d-96c2-4f5a-85c1-418b17de4367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:40.344915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 703/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:40.500369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca9dc5-16ba-4686-aed7-6c3a099d2e3b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:40.515473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca9dc5-16ba-4686-aed7-6c3a099d2e3b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:40.520938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56244cc6-42ba-4abd-a261-f03d7deea0f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:40.521571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56244cc6-42ba-4abd-a261-f03d7deea0f0 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:40.521855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56244cc6-42ba-4abd-a261-f03d7deea0f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:40.522133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56244cc6-42ba-4abd-a261-f03d7deea0f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:40.531219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56244cc6-42ba-4abd-a261-f03d7deea0f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:40.532918 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56244cc6-42ba-4abd-a261-f03d7deea0f0 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:40.542807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 734/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:41.407567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827a498-c955-4ab5-b7bf-15ffac40954f None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:41.408102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827a498-c955-4ab5-b7bf-15ffac40954f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:41.408102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827a498-c955-4ab5-b7bf-15ffac40954f None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:41.409742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 704/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:41.415041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4592da-b1be-443d-9679-4bfa0ea3b0ca None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:41.415472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4592da-b1be-443d-9679-4bfa0ea3b0ca None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:41.415661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4592da-b1be-443d-9679-4bfa0ea3b0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:41.433607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-da4592da-b1be-443d-9679-4bfa0ea3b0ca None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:41.434806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-da4592da-b1be-443d-9679-4bfa0ea3b0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:41.442789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 735/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:41.606687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827a498-c955-4ab5-b7bf-15ffac40954f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:41.621295 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827a498-c955-4ab5-b7bf-15ffac40954f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:41.626677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbd8cc-f16f-47d1-8814-eddf3882cfd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:41.629851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbd8cc-f16f-47d1-8814-eddf3882cfd4 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:41.629851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbd8cc-f16f-47d1-8814-eddf3882cfd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:41.629851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbd8cc-f16f-47d1-8814-eddf3882cfd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:41.636664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbd8cc-f16f-47d1-8814-eddf3882cfd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:41.638235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbd8cc-f16f-47d1-8814-eddf3882cfd4 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:41.647625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 705/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:43.617629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26ec7a-6d21-4898-a65f-3cfd13e827f9 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:43.618212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26ec7a-6d21-4898-a65f-3cfd13e827f9 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:43.618212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26ec7a-6d21-4898-a65f-3cfd13e827f9 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:43.619630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 736/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:43.624820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd67ea-6ec6-4ba7-82d3-d4ffb4d09ad6 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:43.625227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd67ea-6ec6-4ba7-82d3-d4ffb4d09ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:43.625382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd67ea-6ec6-4ba7-82d3-d4ffb4d09ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:43.642876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-04fd67ea-6ec6-4ba7-82d3-d4ffb4d09ad6 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:43.644106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-04fd67ea-6ec6-4ba7-82d3-d4ffb4d09ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:43.652035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 706/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:43.691277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26ec7a-6d21-4898-a65f-3cfd13e827f9 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:43.701944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26ec7a-6d21-4898-a65f-3cfd13e827f9 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:43.705450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff2839-8221-4166-ac89-37daef087427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:43.706089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff2839-8221-4166-ac89-37daef087427 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:43.706369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff2839-8221-4166-ac89-37daef087427 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:43.706650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff2839-8221-4166-ac89-37daef087427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:43.715602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff2839-8221-4166-ac89-37daef087427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:43.717016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff2839-8221-4166-ac89-37daef087427 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:43.726881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 737/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:45.989406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba67d9c-97aa-48df-99b8-d9707bd91515 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:45.989993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba67d9c-97aa-48df-99b8-d9707bd91515 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:45.989993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba67d9c-97aa-48df-99b8-d9707bd91515 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:45.991439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 707/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:45.996835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e930e16-f11e-4f39-ba73-a50595cbc7c4 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:45.997274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e930e16-f11e-4f39-ba73-a50595cbc7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:45.997384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e930e16-f11e-4f39-ba73-a50595cbc7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:46.015675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-1e930e16-f11e-4f39-ba73-a50595cbc7c4 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:46.016881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1e930e16-f11e-4f39-ba73-a50595cbc7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:46.025344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 738/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:46.064527 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba67d9c-97aa-48df-99b8-d9707bd91515 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:46.067063 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3ba67d9c-97aa-48df-99b8-d9707bd91515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:46.079900 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba67d9c-97aa-48df-99b8-d9707bd91515 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:46.083505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00380f0e-b5a3-4b58-aebe-233cea4305ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:46.084218 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00380f0e-b5a3-4b58-aebe-233cea4305ed service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:46.084473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00380f0e-b5a3-4b58-aebe-233cea4305ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:46.084753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00380f0e-b5a3-4b58-aebe-233cea4305ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:46.093906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00380f0e-b5a3-4b58-aebe-233cea4305ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:46.095331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00380f0e-b5a3-4b58-aebe-233cea4305ed service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:46.104853 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 708/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:48.597728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541b94a-c980-4ed5-bb51-83cc031222d4 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:48.598277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541b94a-c980-4ed5-bb51-83cc031222d4 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:48.598277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541b94a-c980-4ed5-bb51-83cc031222d4 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:48.599609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 739/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:48.604809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fdc02-d29e-4d6a-b68b-50d1e4891c44 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:48.605215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fdc02-d29e-4d6a-b68b-50d1e4891c44 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:48.605370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fdc02-d29e-4d6a-b68b-50d1e4891c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:48.623217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7a3fdc02-d29e-4d6a-b68b-50d1e4891c44 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:48.624440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7a3fdc02-d29e-4d6a-b68b-50d1e4891c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:48.632791 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 709/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:48.671246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541b94a-c980-4ed5-bb51-83cc031222d4 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:48.682673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541b94a-c980-4ed5-bb51-83cc031222d4 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:48.686116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7d50f-cc8e-48ed-94d6-a6c5881da38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:48.686744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7d50f-cc8e-48ed-94d6-a6c5881da38e service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:48.687020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7d50f-cc8e-48ed-94d6-a6c5881da38e service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:48.687291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7d50f-cc8e-48ed-94d6-a6c5881da38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:48.696767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7d50f-cc8e-48ed-94d6-a6c5881da38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:48.698175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7d50f-cc8e-48ed-94d6-a6c5881da38e service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:48.707540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 740/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:50.877976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dd4ff-1cbb-4832-8bf7-ca0c9a98d0af None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:50.878548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dd4ff-1cbb-4832-8bf7-ca0c9a98d0af None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:50.878548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dd4ff-1cbb-4832-8bf7-ca0c9a98d0af None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:50.879997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 710/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:50.885360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec95f75-ef8f-438b-a06a-84b8d6b0be3b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:50.885758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec95f75-ef8f-438b-a06a-84b8d6b0be3b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:50.885919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec95f75-ef8f-438b-a06a-84b8d6b0be3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:50.903873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5ec95f75-ef8f-438b-a06a-84b8d6b0be3b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:50.905071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5ec95f75-ef8f-438b-a06a-84b8d6b0be3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:50.913120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 741/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:50.952775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dd4ff-1cbb-4832-8bf7-ca0c9a98d0af None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:50.964178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dd4ff-1cbb-4832-8bf7-ca0c9a98d0af None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:50.967855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fda00-9c1f-4de0-9134-7d096282833d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:50.968501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fda00-9c1f-4de0-9134-7d096282833d service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:50.968779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fda00-9c1f-4de0-9134-7d096282833d service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:50.969055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fda00-9c1f-4de0-9134-7d096282833d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:50.978283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fda00-9c1f-4de0-9134-7d096282833d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:50.979744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fda00-9c1f-4de0-9134-7d096282833d service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:50.988981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 711/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:53.099006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f338f55-0794-49a6-a3c9-af7d0a820a30 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:53.099570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f338f55-0794-49a6-a3c9-af7d0a820a30 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:53.099570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f338f55-0794-49a6-a3c9-af7d0a820a30 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:53.101340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 742/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:53.106735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94314bdf-c709-4018-a86e-3e81a65fb9bd None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:53.107177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94314bdf-c709-4018-a86e-3e81a65fb9bd None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:53.107287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94314bdf-c709-4018-a86e-3e81a65fb9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:53.125404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-94314bdf-c709-4018-a86e-3e81a65fb9bd None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:53.126602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-94314bdf-c709-4018-a86e-3e81a65fb9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:53.135344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 712/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:53.173166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f338f55-0794-49a6-a3c9-af7d0a820a30 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:53.184318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f338f55-0794-49a6-a3c9-af7d0a820a30 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:53.187792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831ae05-157d-4fde-9546-4afbc10d00f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:53.188413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ae05-157d-4fde-9546-4afbc10d00f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:53.188693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ae05-157d-4fde-9546-4afbc10d00f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:53.188967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ae05-157d-4fde-9546-4afbc10d00f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:53.198436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831ae05-157d-4fde-9546-4afbc10d00f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:53.199893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831ae05-157d-4fde-9546-4afbc10d00f6 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:53.209054 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 743/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:55.313173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151e6e9-d816-4407-b893-fa81a589e92f None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:55.313743 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151e6e9-d816-4407-b893-fa81a589e92f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:55.313743 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151e6e9-d816-4407-b893-fa81a589e92f None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:55.315122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 713/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:55.321083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a47e8-4beb-47d6-9f62-c9bea506e869 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:55.321495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a47e8-4beb-47d6-9f62-c9bea506e869 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:55.321637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a47e8-4beb-47d6-9f62-c9bea506e869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:55.339575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-dc2a47e8-4beb-47d6-9f62-c9bea506e869 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:55.340803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-dc2a47e8-4beb-47d6-9f62-c9bea506e869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:55.348714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 744/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:55.387486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151e6e9-d816-4407-b893-fa81a589e92f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:55.398067 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151e6e9-d816-4407-b893-fa81a589e92f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:55.402286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507229d7-6a0f-46c6-9c49-736a6874de52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:55.402919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507229d7-6a0f-46c6-9c49-736a6874de52 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:55.403194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507229d7-6a0f-46c6-9c49-736a6874de52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:55.403468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507229d7-6a0f-46c6-9c49-736a6874de52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:55.412791 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507229d7-6a0f-46c6-9c49-736a6874de52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:55.414205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507229d7-6a0f-46c6-9c49-736a6874de52 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:55.423550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 714/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:57.512757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86062e7-82c3-4b1c-95d5-4d814280a7ff None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:57.513325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86062e7-82c3-4b1c-95d5-4d814280a7ff None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:57.513325 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86062e7-82c3-4b1c-95d5-4d814280a7ff None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:57.514697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 745/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:29:57.520151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e2fd9-cb5e-454f-a079-26996b004ced None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:57.520559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e2fd9-cb5e-454f-a079-26996b004ced None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:57.520755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e2fd9-cb5e-454f-a079-26996b004ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:57.526912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=70 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:29:57.539027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-635e2fd9-cb5e-454f-a079-26996b004ced None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:57.540311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-635e2fd9-cb5e-454f-a079-26996b004ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:57.548206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 715/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:57.586253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86062e7-82c3-4b1c-95d5-4d814280a7ff None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:57.591864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=73 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:29:57.597511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86062e7-82c3-4b1c-95d5-4d814280a7ff None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:57.601195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd8c78-6534-4a12-9d8d-d519590a21fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:57.619477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd8c78-6534-4a12-9d8d-d519590a21fd service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:57.619477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd8c78-6534-4a12-9d8d-d519590a21fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:57.619477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd8c78-6534-4a12-9d8d-d519590a21fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:57.619477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd8c78-6534-4a12-9d8d-d519590a21fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:57.619477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd8c78-6534-4a12-9d8d-d519590a21fd service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:57.623024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 746/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:29:59.705212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74e974-9415-4833-b4bf-40f303de18d4 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:59.705730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74e974-9415-4833-b4bf-40f303de18d4 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:59.705730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74e974-9415-4833-b4bf-40f303de18d4 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:59.707061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 716/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:29:59.712482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f9605-2086-4e5a-9198-6fe9a882d303 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:59.712910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f9605-2086-4e5a-9198-6fe9a882d303 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:59.713121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f9605-2086-4e5a-9198-6fe9a882d303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:59.730681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-b71f9605-2086-4e5a-9198-6fe9a882d303 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:29:59.731940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b71f9605-2086-4e5a-9198-6fe9a882d303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:29:59.739952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 747/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:29:59.778782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74e974-9415-4833-b4bf-40f303de18d4 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:29:59.789878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74e974-9415-4833-b4bf-40f303de18d4 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:29:59.793634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3497359-008d-4fbc-8a47-af4d3d539da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:29:59.794270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3497359-008d-4fbc-8a47-af4d3d539da5 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:29:59.794609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3497359-008d-4fbc-8a47-af4d3d539da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:29:59.794834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3497359-008d-4fbc-8a47-af4d3d539da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:29:59.804315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3497359-008d-4fbc-8a47-af4d3d539da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:29:59.805737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3497359-008d-4fbc-8a47-af4d3d539da5 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:29:59.815072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 717/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:30:01.907478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d16ed-1229-4b01-a35c-58d6c789f7e4 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:01.907977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d16ed-1229-4b01-a35c-58d6c789f7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:01.908086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d16ed-1229-4b01-a35c-58d6c789f7e4 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:01.909695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 748/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:01.915662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7dbec-5ea2-46fc-b5a0-3bed52219cc1 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:01.916131 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7dbec-5ea2-46fc-b5a0-3bed52219cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:01.916254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7dbec-5ea2-46fc-b5a0-3bed52219cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:01.933894 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9ee7dbec-5ea2-46fc-b5a0-3bed52219cc1 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:01.935111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9ee7dbec-5ea2-46fc-b5a0-3bed52219cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:01.942962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 718/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:01.980426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d16ed-1229-4b01-a35c-58d6c789f7e4 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:30:01.991586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d16ed-1229-4b01-a35c-58d6c789f7e4 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:30:01.995209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fe0c7-2307-4825-87bb-8c7bb005eda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:30:01.995880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fe0c7-2307-4825-87bb-8c7bb005eda9 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:01.996165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fe0c7-2307-4825-87bb-8c7bb005eda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:01.996440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fe0c7-2307-4825-87bb-8c7bb005eda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:02.006324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fe0c7-2307-4825-87bb-8c7bb005eda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:30:02.007947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fe0c7-2307-4825-87bb-8c7bb005eda9 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:30:02.017819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 749/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:30:04.079188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05420be-5b77-41c6-8ee3-f54060b9b73e None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:04.079810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05420be-5b77-41c6-8ee3-f54060b9b73e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:04.079810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05420be-5b77-41c6-8ee3-f54060b9b73e None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:04.081191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 719/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:04.087013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41511da9-0afb-4fd7-b8ba-4d02fea2f190 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:04.087425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41511da9-0afb-4fd7-b8ba-4d02fea2f190 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:04.087602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41511da9-0afb-4fd7-b8ba-4d02fea2f190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:04.105902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-41511da9-0afb-4fd7-b8ba-4d02fea2f190 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:04.107110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-41511da9-0afb-4fd7-b8ba-4d02fea2f190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:04.114979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 750/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:04.151260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05420be-5b77-41c6-8ee3-f54060b9b73e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:30:04.166953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05420be-5b77-41c6-8ee3-f54060b9b73e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:30:04.170539 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57a0a4-ff4d-4c0b-9b40-0439741694c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:30:04.171175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57a0a4-ff4d-4c0b-9b40-0439741694c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:04.171459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57a0a4-ff4d-4c0b-9b40-0439741694c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:04.171771 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57a0a4-ff4d-4c0b-9b40-0439741694c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:04.180871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57a0a4-ff4d-4c0b-9b40-0439741694c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:30:04.182299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57a0a4-ff4d-4c0b-9b40-0439741694c1 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:30:04.192076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 720/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:30:14.100162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=17 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:30:14.187716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=17 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:30:24.204816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb66ef2-3733-403d-ab60-eeb635784024 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:24.205345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb66ef2-3733-403d-ab60-eeb635784024 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:24.205345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb66ef2-3733-403d-ab60-eeb635784024 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:24.206680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 751/1471] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:24.213009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b28a9-1c88-4299-9fae-c940429f51c9 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:24.213437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b28a9-1c88-4299-9fae-c940429f51c9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:24.213576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b28a9-1c88-4299-9fae-c940429f51c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:24.232544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e95b28a9-1c88-4299-9fae-c940429f51c9 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:24.233766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e95b28a9-1c88-4299-9fae-c940429f51c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:24.241852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 721/1472] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:30:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:24.291083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cebc54-fd65-4d59-9e36-78f1ad5a98d7 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:24.291671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cebc54-fd65-4d59-9e36-78f1ad5a98d7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:24.292173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cebc54-fd65-4d59-9e36-78f1ad5a98d7 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:24.293723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 752/1473] 10.222.0.64 () {54 vars in 962 bytes} [Mon Jun 27 09:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:24.298719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd36576-7610-4c98-a786-92c538c26e2a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:24.299013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd36576-7610-4c98-a786-92c538c26e2a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:24.299295 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd36576-7610-4c98-a786-92c538c26e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:24.319489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-5fd36576-7610-4c98-a786-92c538c26e2a None None] MFA Rules not processed for user `e33d1473077247cc97dcfd99b3da5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:24.320736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5fd36576-7610-4c98-a786-92c538c26e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:24.327466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 722/1474] 10.222.0.64 () {58 vars in 1059 bytes} [Mon Jun 27 09:30:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:24.334916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cebc54-fd65-4d59-9e36-78f1ad5a98d7 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:30:24.346262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cebc54-fd65-4d59-9e36-78f1ad5a98d7 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:30:24.350540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fe55b-bda4-45eb-92fe-5d8fa242ae6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e33d1473077247cc97dcfd99b3da5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:30:24.351174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe55b-bda4-45eb-92fe-5d8fa242ae6b service glance] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:24.351477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe55b-bda4-45eb-92fe-5d8fa242ae6b service glance] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:24.351796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe55b-bda4-45eb-92fe-5d8fa242ae6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:24.361112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fe55b-bda4-45eb-92fe-5d8fa242ae6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:30:24.362541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fe55b-bda4-45eb-92fe-5d8fa242ae6b service glance] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:30:24.372953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 753/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:30:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:30:26.512150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d572835-0f14-4c01-8c2b-796364ebee7e None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:26.512710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d572835-0f14-4c01-8c2b-796364ebee7e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:26.512710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d572835-0f14-4c01-8c2b-796364ebee7e None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:26.514113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 723/1476] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:26.519404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1fec4-5135-459f-bf31-bdb8c413b28e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:26.519860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1fec4-5135-459f-bf31-bdb8c413b28e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:26.520027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1fec4-5135-459f-bf31-bdb8c413b28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:26.539850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-56d1fec4-5135-459f-bf31-bdb8c413b28e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:26.540991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-56d1fec4-5135-459f-bf31-bdb8c413b28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:26.548911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 754/1477] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:26.573861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d572835-0f14-4c01-8c2b-796364ebee7e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:30:26.586468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d572835-0f14-4c01-8c2b-796364ebee7e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:30:26.590940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aa0e7-9dd8-4438-a2cd-7ed91de23304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:30:26.591626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa0e7-9dd8-4438-a2cd-7ed91de23304 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:26.591910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa0e7-9dd8-4438-a2cd-7ed91de23304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:26.592183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa0e7-9dd8-4438-a2cd-7ed91de23304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:26.592913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aa0e7-9dd8-4438-a2cd-7ed91de23304 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:30:26.594679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING py.warnings [None req-ff3aa0e7-9dd8-4438-a2cd-7ed91de23304 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:30:26.594679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: warnings.warn(msg) Jun 27 09:30:26.594679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: Jun 27 09:30:26.595864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aa0e7-9dd8-4438-a2cd-7ed91de23304 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:30:26.614688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 724/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Mon Jun 27 09:30:26 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:30:28.711465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9e177-37b7-46f3-9d33-791e1c0545ad None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:28.712027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9e177-37b7-46f3-9d33-791e1c0545ad None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:28.712027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9e177-37b7-46f3-9d33-791e1c0545ad None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:28.713454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 755/1479] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:28.718806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921861f-9d07-4527-9a7c-814af91e210c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:28.719269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921861f-9d07-4527-9a7c-814af91e210c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:28.719417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921861f-9d07-4527-9a7c-814af91e210c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:28.736822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4921861f-9d07-4527-9a7c-814af91e210c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:28.738009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4921861f-9d07-4527-9a7c-814af91e210c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:28.745752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 725/1480] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:28.789234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9e177-37b7-46f3-9d33-791e1c0545ad None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:30:28.803877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9e177-37b7-46f3-9d33-791e1c0545ad None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:30:28.810835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a4e0a-9afc-41b4-9a3a-11329c78bb28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:30:28.811477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a4e0a-9afc-41b4-9a3a-11329c78bb28 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:28.811804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a4e0a-9afc-41b4-9a3a-11329c78bb28 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:28.812082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a4e0a-9afc-41b4-9a3a-11329c78bb28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:28.821495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a4e0a-9afc-41b4-9a3a-11329c78bb28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:30:28.823062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a4e0a-9afc-41b4-9a3a-11329c78bb28 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:30:28.832835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 756/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:30:31.073836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c300f-7857-4200-b21e-8a6ac927ef27 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:31.074343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c300f-7857-4200-b21e-8a6ac927ef27 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:31.074450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c300f-7857-4200-b21e-8a6ac927ef27 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:31.075866 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 726/1482] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:31.081452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e6ee1-901a-4a15-8916-225cd210ec50 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:31.081882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e6ee1-901a-4a15-8916-225cd210ec50 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:31.082214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e6ee1-901a-4a15-8916-225cd210ec50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:31.102165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a77e6ee1-901a-4a15-8916-225cd210ec50 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:31.103399 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a77e6ee1-901a-4a15-8916-225cd210ec50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:31.112068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 757/1483] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:31.150651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c300f-7857-4200-b21e-8a6ac927ef27 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:30:31.163717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c300f-7857-4200-b21e-8a6ac927ef27 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:30:31.167950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd944c-5259-49f1-b2ae-8293ac2cb470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:30:31.168465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd944c-5259-49f1-b2ae-8293ac2cb470 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:31.168763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd944c-5259-49f1-b2ae-8293ac2cb470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:31.169044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd944c-5259-49f1-b2ae-8293ac2cb470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:31.179089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd944c-5259-49f1-b2ae-8293ac2cb470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:30:31.180663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd944c-5259-49f1-b2ae-8293ac2cb470 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:30:31.190529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 727/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:30:33.330154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630302ca-5506-4c07-b883-4e6c869d6839 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:33.330701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630302ca-5506-4c07-b883-4e6c869d6839 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:33.330815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630302ca-5506-4c07-b883-4e6c869d6839 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:33.332154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 758/1485] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:33.337910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee83f0-3116-4af1-a0cd-4f6c3582851a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:33.338348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee83f0-3116-4af1-a0cd-4f6c3582851a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:33.338548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee83f0-3116-4af1-a0cd-4f6c3582851a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:33.357351 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d7ee83f0-3116-4af1-a0cd-4f6c3582851a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:33.358634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d7ee83f0-3116-4af1-a0cd-4f6c3582851a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:33.367744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 728/1486] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:33.407840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630302ca-5506-4c07-b883-4e6c869d6839 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:30:33.421967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630302ca-5506-4c07-b883-4e6c869d6839 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:30:33.425774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d33ef-6f23-463f-b52a-c3d7b8ae0248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:30:33.426500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d33ef-6f23-463f-b52a-c3d7b8ae0248 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:33.426786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d33ef-6f23-463f-b52a-c3d7b8ae0248 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:33.427573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d33ef-6f23-463f-b52a-c3d7b8ae0248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:33.437600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d33ef-6f23-463f-b52a-c3d7b8ae0248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:30:33.439036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d33ef-6f23-463f-b52a-c3d7b8ae0248 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:30:33.450061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 759/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:30:35.620158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870ac08-9e0b-4348-863d-416d352068a5 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:35.620666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870ac08-9e0b-4348-863d-416d352068a5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:35.620666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a870ac08-9e0b-4348-863d-416d352068a5 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:35.622189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 729/1488] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:35.627351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac18d3-9927-4bd1-b2f5-e5135bcc789f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:35.627794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac18d3-9927-4bd1-b2f5-e5135bcc789f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:35.627944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac18d3-9927-4bd1-b2f5-e5135bcc789f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:35.650652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-09ac18d3-9927-4bd1-b2f5-e5135bcc789f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:35.660694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-09ac18d3-9927-4bd1-b2f5-e5135bcc789f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:35.660694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 760/1489] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:35.702585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870ac08-9e0b-4348-863d-416d352068a5 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:30:35.713756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a870ac08-9e0b-4348-863d-416d352068a5 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:30:35.717399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308f9a1-8145-4d6d-802c-d0a7a0c75417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:30:35.718155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308f9a1-8145-4d6d-802c-d0a7a0c75417 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:35.718437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308f9a1-8145-4d6d-802c-d0a7a0c75417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:35.718710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308f9a1-8145-4d6d-802c-d0a7a0c75417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:35.728215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308f9a1-8145-4d6d-802c-d0a7a0c75417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:30:35.729637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308f9a1-8145-4d6d-802c-d0a7a0c75417 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:30:35.739419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 730/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:30:38.297258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9ce2f-517e-4123-aaa2-34c99b6e20bc None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:38.297804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9ce2f-517e-4123-aaa2-34c99b6e20bc None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:38.297804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9ce2f-517e-4123-aaa2-34c99b6e20bc None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:38.299306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 761/1491] 10.222.0.64 () {54 vars in 926 bytes} [Mon Jun 27 09:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:30:38.305036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a668c-91cc-416b-ad28-126ad4646c88 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:38.305525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a668c-91cc-416b-ad28-126ad4646c88 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:38.305615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a668c-91cc-416b-ad28-126ad4646c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:38.324616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a07a668c-91cc-416b-ad28-126ad4646c88 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:30:38.325813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a07a668c-91cc-416b-ad28-126ad4646c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:30:38.334004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 731/1492] 10.222.0.64 () {58 vars in 1023 bytes} [Mon Jun 27 09:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:30:38.372724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9ce2f-517e-4123-aaa2-34c99b6e20bc None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:30:38.384308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9ce2f-517e-4123-aaa2-34c99b6e20bc None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:30:38.387831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e10c24-e840-4dcf-956e-6e5141a9742d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:30:38.388498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e10c24-e840-4dcf-956e-6e5141a9742d service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:30:38.388781 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e10c24-e840-4dcf-956e-6e5141a9742d service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:30:38.389059 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e10c24-e840-4dcf-956e-6e5141a9742d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:30:38.398024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e10c24-e840-4dcf-956e-6e5141a9742d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:30:38.399639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e10c24-e840-4dcf-956e-6e5141a9742d service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:30:38.409230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 762/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:30:48.319262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=33 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:30:48.404904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=33 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:32:12.153550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4f04f-8911-4334-870a-a3c099b1ba82 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.154143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4f04f-8911-4334-870a-a3c099b1ba82 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.154143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4f04f-8911-4334-870a-a3c099b1ba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.172728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-97d4f04f-8911-4334-870a-a3c099b1ba82 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:32:12.173942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-97d4f04f-8911-4334-870a-a3c099b1ba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:32:12.182582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 732/1494] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:32:12.190574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a006f-a6c3-4065-b06e-dd986cfa45ec None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.191080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a006f-a6c3-4065-b06e-dd986cfa45ec None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.191189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a006f-a6c3-4065-b06e-dd986cfa45ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.208772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ec3a006f-a6c3-4065-b06e-dd986cfa45ec None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:32:12.210459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ec3a006f-a6c3-4065-b06e-dd986cfa45ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:32:12.218230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 763/1495] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:32:12.222918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4f04f-8911-4334-870a-a3c099b1ba82 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:12.233101 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4f04f-8911-4334-870a-a3c099b1ba82 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:12.237601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:12.238308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.238658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.238939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.239703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:32:12.240012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:12.241017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING py.warnings [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:32:12.241017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: warnings.warn(msg) Jun 27 09:32:12.241017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: Jun 27 09:32:12.241935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:12.254132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 733/1496] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:32:12.262177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a006f-a6c3-4065-b06e-dd986cfa45ec None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:12.272340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a006f-a6c3-4065-b06e-dd986cfa45ec None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:12.277789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b430e3-8f61-441c-b502-88602cf95873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:12.278452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b430e3-8f61-441c-b502-88602cf95873 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.278734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b430e3-8f61-441c-b502-88602cf95873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.279008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b430e3-8f61-441c-b502-88602cf95873 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.279741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b430e3-8f61-441c-b502-88602cf95873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:12.281549 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: WARNING py.warnings [None req-f4b430e3-8f61-441c-b502-88602cf95873 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:32:12.281549 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: warnings.warn(msg) Jun 27 09:32:12.281549 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: Jun 27 09:32:12.282748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b430e3-8f61-441c-b502-88602cf95873 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:12.417733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 764/1497] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:32:12 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:32:12.422246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:12.432237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032aa9f-f630-4837-8514-11309d5a669f admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:12.486059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777902d-0f23-417c-935f-dbd6e2f103a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:12.486718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777902d-0f23-417c-935f-dbd6e2f103a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.487003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777902d-0f23-417c-935f-dbd6e2f103a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.487275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777902d-0f23-417c-935f-dbd6e2f103a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.487988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777902d-0f23-417c-935f-dbd6e2f103a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:12.488986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING py.warnings [None req-9777902d-0f23-417c-935f-dbd6e2f103a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:32:12.488986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: warnings.warn(msg) Jun 27 09:32:12.488986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: Jun 27 09:32:12.490187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777902d-0f23-417c-935f-dbd6e2f103a2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:12.592779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 734/1498] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:32:12 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:32:12.597432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b430e3-8f61-441c-b502-88602cf95873 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:12.607947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b430e3-8f61-441c-b502-88602cf95873 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:12.612546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69f4e1-6316-4524-8994-ff6dd1e1851c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:12.613260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69f4e1-6316-4524-8994-ff6dd1e1851c admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.613548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69f4e1-6316-4524-8994-ff6dd1e1851c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.613826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69f4e1-6316-4524-8994-ff6dd1e1851c admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.614582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69f4e1-6316-4524-8994-ff6dd1e1851c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:12.615644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: WARNING py.warnings [None req-7b69f4e1-6316-4524-8994-ff6dd1e1851c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:32:12.615644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: warnings.warn(msg) Jun 27 09:32:12.615644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: Jun 27 09:32:12.616517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69f4e1-6316-4524-8994-ff6dd1e1851c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:12.629631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 765/1499] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:32:12.634955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777902d-0f23-417c-935f-dbd6e2f103a2 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:12.645051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777902d-0f23-417c-935f-dbd6e2f103a2 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:12.650328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecca2a5-09ef-429c-96fa-53bd9b8240c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:12.650981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecca2a5-09ef-429c-96fa-53bd9b8240c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.651264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecca2a5-09ef-429c-96fa-53bd9b8240c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.651548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecca2a5-09ef-429c-96fa-53bd9b8240c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.652292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecca2a5-09ef-429c-96fa-53bd9b8240c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:12.653289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING py.warnings [None req-4ecca2a5-09ef-429c-96fa-53bd9b8240c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:32:12.653289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: warnings.warn(msg) Jun 27 09:32:12.653289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: Jun 27 09:32:12.654237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecca2a5-09ef-429c-96fa-53bd9b8240c7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:12.668162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 735/1500] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:32:12.673533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69f4e1-6316-4524-8994-ff6dd1e1851c admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:12.683724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69f4e1-6316-4524-8994-ff6dd1e1851c admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:12.688494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22c7a4-ee65-4a87-9a0e-6377241960bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:12.689377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22c7a4-ee65-4a87-9a0e-6377241960bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.689659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22c7a4-ee65-4a87-9a0e-6377241960bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.689945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22c7a4-ee65-4a87-9a0e-6377241960bd admin admin] PATH_INFO: `/v3/projects/c5372a03be224a8c9920b224ad9ff487/users/f51f77fe0cb34c3fb5054185eae0cfe6/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.705636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22c7a4-ee65-4a87-9a0e-6377241960bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5372a03be224a8c9920b224ad9ff487, user_id=f51f77fe0cb34c3fb5054185eae0cfe6, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:12.706667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: WARNING py.warnings [None req-7c22c7a4-ee65-4a87-9a0e-6377241960bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:32:12.706667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: warnings.warn(msg) Jun 27 09:32:12.706667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: Jun 27 09:32:12.708652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22c7a4-ee65-4a87-9a0e-6377241960bd admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:12.779665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 766/1501] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:32:12 2022] PUT /identity/v3/projects/c5372a03be224a8c9920b224ad9ff487/users/f51f77fe0cb34c3fb5054185eae0cfe6/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:32:12.791405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3faa55-358e-4efc-ba1c-d18fcada7044 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.791731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3faa55-358e-4efc-ba1c-d18fcada7044 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.792031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3faa55-358e-4efc-ba1c-d18fcada7044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.818251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4a3faa55-358e-4efc-ba1c-d18fcada7044 None None] MFA Rules not processed for user `f51f77fe0cb34c3fb5054185eae0cfe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:32:12.819986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4a3faa55-358e-4efc-ba1c-d18fcada7044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:32:12.882425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 736/1502] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:32:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:32:12.895719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7088-922f-46ec-b40f-d693f0458309 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.896019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7088-922f-46ec-b40f-d693f0458309 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.896306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7088-922f-46ec-b40f-d693f0458309 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.898318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 767/1503] 10.222.0.64 () {54 vars in 963 bytes} [Mon Jun 27 09:32:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:32:12.903057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32826440-0733-4dc0-860c-4919dd6106de None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.903352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32826440-0733-4dc0-860c-4919dd6106de None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.903666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32826440-0733-4dc0-860c-4919dd6106de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:12.921414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-32826440-0733-4dc0-860c-4919dd6106de None None] MFA Rules not processed for user `8c8abb981d1b4c82a43b5a3372f98398`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:32:12.922633 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-32826440-0733-4dc0-860c-4919dd6106de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:32:12.966018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:32:12.973395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 737/1504] 10.222.0.64 () {58 vars in 1060 bytes} [Mon Jun 27 09:32:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:32:12.981152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b7088-922f-46ec-b40f-d693f0458309 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:12.992191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b7088-922f-46ec-b40f-d693f0458309 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:12.995733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe6f3-f6e4-4f12-ad79-79cecb92911c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:12.996434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe6f3-f6e4-4f12-ad79-79cecb92911c service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:12.996721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe6f3-f6e4-4f12-ad79-79cecb92911c service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:12.996998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe6f3-f6e4-4f12-ad79-79cecb92911c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:13.005699 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afe6f3-f6e4-4f12-ad79-79cecb92911c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:13.007763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afe6f3-f6e4-4f12-ad79-79cecb92911c service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:13.016739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 768/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:32:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:32:13.058567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0810d83-1f5b-46b0-b7a0-0b9325dc0131 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:13.058862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0810d83-1f5b-46b0-b7a0-0b9325dc0131 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:13.059141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0810d83-1f5b-46b0-b7a0-0b9325dc0131 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:13.060460 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 738/1506] 10.222.0.64 () {52 vars in 829 bytes} [Mon Jun 27 09:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:32:13.064204 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe6f3-f6e4-4f12-ad79-79cecb92911c service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:13.125071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe6f3-f6e4-4f12-ad79-79cecb92911c service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:13.129640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:13.130527 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:13.130793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:13.131712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] PATH_INFO: `/v3/users/f51f77fe0cb34c3fb5054185eae0cfe6` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:13.132903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f51f77fe0cb34c3fb5054185eae0cfe6)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:13.133917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: WARNING py.warnings [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:32:13.133917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: warnings.warn(msg) Jun 27 09:32:13.133917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: Jun 27 09:32:13.135126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:13.325502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f51f77fe0cb34c3fb5054185eae0cfe6'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:13.402101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f51f77fe0cb34c3fb5054185eae0cfe6'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:13.402852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f51f77fe0cb34c3fb5054185eae0cfe6'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:13.416837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f51f77fe0cb34c3fb5054185eae0cfe6'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:13.421639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f51f77fe0cb34c3fb5054185eae0cfe6'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:13.434975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:32:13.442667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 769/1507] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:32:13 2022] DELETE /identity/v3/users/f51f77fe0cb34c3fb5054185eae0cfe6 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:32:13.447418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64671a-d2d0-48ee-98d4-a1c3b7574203 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:13.456951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64671a-d2d0-48ee-98d4-a1c3b7574203 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:13.456951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64671a-d2d0-48ee-98d4-a1c3b7574203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:13.466487 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-6c64671a-d2d0-48ee-98d4-a1c3b7574203 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:32:13.467737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6c64671a-d2d0-48ee-98d4-a1c3b7574203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:32:13.530842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 739/1508] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:32:13.542095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:13.543584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:32:13.601273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7da41-35c9-46dd-ad02-b7ef156e134a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:13.604766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912c26c-0cdc-421f-9d02-cd61a446b763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:13.605414 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912c26c-0cdc-421f-9d02-cd61a446b763 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:13.605700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912c26c-0cdc-421f-9d02-cd61a446b763 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:13.606113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912c26c-0cdc-421f-9d02-cd61a446b763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:13.615220 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912c26c-0cdc-421f-9d02-cd61a446b763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:13.617324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912c26c-0cdc-421f-9d02-cd61a446b763 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:13.626755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 770/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:32:14.068950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64671a-d2d0-48ee-98d4-a1c3b7574203 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:32:14.070662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6c64671a-d2d0-48ee-98d4-a1c3b7574203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:32:14.082530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64671a-d2d0-48ee-98d4-a1c3b7574203 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:32:14.087709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:32:14.088605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:32:14.088887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:32:14.089164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] PATH_INFO: `/v3/projects/c5372a03be224a8c9920b224ad9ff487` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:32:14.090359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5372a03be224a8c9920b224ad9ff487)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:32:14.091450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING py.warnings [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:32:14.091450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: warnings.warn(msg) Jun 27 09:32:14.091450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: Jun 27 09:32:14.092610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:32:14.208926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5372a03be224a8c9920b224ad9ff487', 'request_id': 'req-7400ab65-5592-47d1-94a9-b1e3c411d69a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:14.215744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5372a03be224a8c9920b224ad9ff487', 'request_id': 'req-7400ab65-5592-47d1-94a9-b1e3c411d69a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:14.229045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5372a03be224a8c9920b224ad9ff487', 'request_id': 'req-7400ab65-5592-47d1-94a9-b1e3c411d69a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:14.294342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] The token cache is being invalidate because project c5372a03be224a8c9920b224ad9ff487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:32:14.294636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:14.295263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7400ab65-5592-47d1-94a9-b1e3c411d69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:32:14.296968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 740/1510] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:32:14 2022] DELETE /identity/v3/projects/c5372a03be224a8c9920b224ad9ff487 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:32:23.622655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:32:24.291226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:33:57.443992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63932dc-4d81-43e6-af2f-86329d8af59e None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:33:57.444541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63932dc-4d81-43e6-af2f-86329d8af59e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:33:57.444541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63932dc-4d81-43e6-af2f-86329d8af59e None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:33:57.445940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 771/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:33:57.451247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579460b8-4888-4b74-b54a-0d1e9d99ce23 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:33:57.451754 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579460b8-4888-4b74-b54a-0d1e9d99ce23 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:33:57.451860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579460b8-4888-4b74-b54a-0d1e9d99ce23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:33:57.470818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-579460b8-4888-4b74-b54a-0d1e9d99ce23 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:33:57.472073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-579460b8-4888-4b74-b54a-0d1e9d99ce23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:33:57.534406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 741/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:33:57.689209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63932dc-4d81-43e6-af2f-86329d8af59e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:33:57.690939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a63932dc-4d81-43e6-af2f-86329d8af59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:33:57.767073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63932dc-4d81-43e6-af2f-86329d8af59e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:33:57.772460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535d61f-c1b9-4a80-a50d-f8654f9c5d72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:33:57.773384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535d61f-c1b9-4a80-a50d-f8654f9c5d72 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:33:57.773665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535d61f-c1b9-4a80-a50d-f8654f9c5d72 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:33:57.773939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535d61f-c1b9-4a80-a50d-f8654f9c5d72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:33:57.782253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535d61f-c1b9-4a80-a50d-f8654f9c5d72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:33:57.783944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535d61f-c1b9-4a80-a50d-f8654f9c5d72 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:33:57.793547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 772/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:33:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:33:58.632097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c8db-1398-40d2-8bed-9be9b5ab2ac7 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:33:58.632632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c8db-1398-40d2-8bed-9be9b5ab2ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:33:58.632759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5c8db-1398-40d2-8bed-9be9b5ab2ac7 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:33:58.634091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 742/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:33:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:33:58.639061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a708ab2-7b56-4090-b889-cd5391a8f513 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:33:58.639506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a708ab2-7b56-4090-b889-cd5391a8f513 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:33:58.639649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a708ab2-7b56-4090-b889-cd5391a8f513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:33:58.660138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3a708ab2-7b56-4090-b889-cd5391a8f513 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:33:58.661326 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3a708ab2-7b56-4090-b889-cd5391a8f513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:33:58.669208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 773/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:33:58.824038 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5c8db-1398-40d2-8bed-9be9b5ab2ac7 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:33:58.839039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5c8db-1398-40d2-8bed-9be9b5ab2ac7 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:33:58.846855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2902de-8e01-4911-8d11-48d6a678d915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:33:58.847568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2902de-8e01-4911-8d11-48d6a678d915 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:33:58.847865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2902de-8e01-4911-8d11-48d6a678d915 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:33:58.848140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2902de-8e01-4911-8d11-48d6a678d915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:33:58.858377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2902de-8e01-4911-8d11-48d6a678d915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:33:58.860891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2902de-8e01-4911-8d11-48d6a678d915 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:33:58.872073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 743/1516] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:33:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:33:59.216284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc746d2-6476-4b70-8efc-a753d6d42f1c None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:33:59.216477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc746d2-6476-4b70-8efc-a753d6d42f1c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:33:59.216759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc746d2-6476-4b70-8efc-a753d6d42f1c None None] PATH_INFO: `/v3` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:33:59.218078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 774/1517] 10.222.0.55 () {54 vars in 925 bytes} [Mon Jun 27 09:33:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:33:59.224448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2c880-f806-413f-878e-91eaa6dd0864 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:33:59.224744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2c880-f806-413f-878e-91eaa6dd0864 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:33:59.225027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2c880-f806-413f-878e-91eaa6dd0864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:33:59.243320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8bf2c880-f806-413f-878e-91eaa6dd0864 None None] MFA Rules not processed for user `19d4d5fa82dc42e498fb22349921885e`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:33:59.244573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8bf2c880-f806-413f-878e-91eaa6dd0864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:33:59.307175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 744/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Mon Jun 27 09:33:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:34:03.341625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba49ee2-6859-4ca6-a8e8-14fef8638cbc None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:03.342167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba49ee2-6859-4ca6-a8e8-14fef8638cbc None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:03.342167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba49ee2-6859-4ca6-a8e8-14fef8638cbc None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:03.343516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 775/1519] 10.222.0.64 () {54 vars in 960 bytes} [Mon Jun 27 09:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:34:03.348846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff82bc-4667-4900-bb65-0787d62204dc None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:03.349803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff82bc-4667-4900-bb65-0787d62204dc None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:03.349936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff82bc-4667-4900-bb65-0787d62204dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:03.368770 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-06ff82bc-4667-4900-bb65-0787d62204dc None None] MFA Rules not processed for user `19d4d5fa82dc42e498fb22349921885e`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:34:03.369998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-06ff82bc-4667-4900-bb65-0787d62204dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:03.378735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 745/1520] 10.222.0.64 () {58 vars in 1057 bytes} [Mon Jun 27 09:34:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:34:03.386618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba49ee2-6859-4ca6-a8e8-14fef8638cbc None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:34:03.400232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba49ee2-6859-4ca6-a8e8-14fef8638cbc None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:34:03.405243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e9597-0dae-47bd-a1a7-c235ea746d4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:34:03.405916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e9597-0dae-47bd-a1a7-c235ea746d4c service placement] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:03.406193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e9597-0dae-47bd-a1a7-c235ea746d4c service placement] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:03.406472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e9597-0dae-47bd-a1a7-c235ea746d4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:03.415603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e9597-0dae-47bd-a1a7-c235ea746d4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:34:03.417179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e9597-0dae-47bd-a1a7-c235ea746d4c service placement] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:34:03.426751 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 776/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:34:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:34:10.094428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1f975-9716-4627-b6a9-293094df3c36 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:10.095003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1f975-9716-4627-b6a9-293094df3c36 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:10.095003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1f975-9716-4627-b6a9-293094df3c36 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:10.096409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 746/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:34:10.101769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df3fc-4148-40fe-9834-e203781ebdc6 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:10.102201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df3fc-4148-40fe-9834-e203781ebdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:10.102333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df3fc-4148-40fe-9834-e203781ebdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:10.120942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ad3df3fc-4148-40fe-9834-e203781ebdc6 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:34:10.122136 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ad3df3fc-4148-40fe-9834-e203781ebdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:10.129977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 777/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:34:10.284626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1f975-9716-4627-b6a9-293094df3c36 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:34:10.300194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1f975-9716-4627-b6a9-293094df3c36 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:34:10.305597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b904f-04a9-4058-ada4-f6ed2a83f0fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:34:10.306217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b904f-04a9-4058-ada4-f6ed2a83f0fb service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:10.306497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b904f-04a9-4058-ada4-f6ed2a83f0fb service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:10.306774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b904f-04a9-4058-ada4-f6ed2a83f0fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:10.315483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b904f-04a9-4058-ada4-f6ed2a83f0fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:34:10.317291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b904f-04a9-4058-ada4-f6ed2a83f0fb service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:34:10.326904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 747/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:34:11.153553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed020a22-8a37-42f8-8519-534341fcfc88 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:11.154102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed020a22-8a37-42f8-8519-534341fcfc88 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:11.154102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed020a22-8a37-42f8-8519-534341fcfc88 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:11.155504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 778/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 09:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:34:11.160385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197af3e-678c-489a-b73f-2270f3a9228b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:11.160830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197af3e-678c-489a-b73f-2270f3a9228b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:11.161050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197af3e-678c-489a-b73f-2270f3a9228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:11.179670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-5197af3e-678c-489a-b73f-2270f3a9228b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:34:11.180864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5197af3e-678c-489a-b73f-2270f3a9228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:11.188743 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 748/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 09:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:34:11.341997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed020a22-8a37-42f8-8519-534341fcfc88 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:34:11.356632 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed020a22-8a37-42f8-8519-534341fcfc88 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:34:11.361972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf5ef3-45e7-4ae0-a1ec-197ab73d5b79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:34:11.362608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf5ef3-45e7-4ae0-a1ec-197ab73d5b79 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:11.362885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf5ef3-45e7-4ae0-a1ec-197ab73d5b79 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:11.363156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf5ef3-45e7-4ae0-a1ec-197ab73d5b79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:11.371939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf5ef3-45e7-4ae0-a1ec-197ab73d5b79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:34:11.373496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf5ef3-45e7-4ae0-a1ec-197ab73d5b79 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:34:11.383357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 779/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:34:16.165686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5db56b-e6f1-41c3-968c-f3c3d52095ae None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:16.166308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5db56b-e6f1-41c3-968c-f3c3d52095ae None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:16.166308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5db56b-e6f1-41c3-968c-f3c3d52095ae None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:16.167721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 749/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:34:16.173411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ebe6a-231d-445a-b3fb-807fa02a6a03 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:16.173839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ebe6a-231d-445a-b3fb-807fa02a6a03 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:16.173973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ebe6a-231d-445a-b3fb-807fa02a6a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:16.191864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-804ebe6a-231d-445a-b3fb-807fa02a6a03 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:34:16.193569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-804ebe6a-231d-445a-b3fb-807fa02a6a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:16.201860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 780/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:34:16.240529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5db56b-e6f1-41c3-968c-f3c3d52095ae None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:34:16.242270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-de5db56b-e6f1-41c3-968c-f3c3d52095ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:16.300126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5db56b-e6f1-41c3-968c-f3c3d52095ae None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:34:16.304192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cf5f8-f06f-4f51-a57a-e8c8b9a6e90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:34:16.304838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cf5f8-f06f-4f51-a57a-e8c8b9a6e90f service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:16.305122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cf5f8-f06f-4f51-a57a-e8c8b9a6e90f service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:16.305397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cf5f8-f06f-4f51-a57a-e8c8b9a6e90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:16.315050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cf5f8-f06f-4f51-a57a-e8c8b9a6e90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:34:16.316503 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cf5f8-f06f-4f51-a57a-e8c8b9a6e90f service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:34:16.322055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:34:16.326790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 750/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:34:18.369605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b061ce-b40f-4fe9-988b-e2209355c15a None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:18.370309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b061ce-b40f-4fe9-988b-e2209355c15a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:18.370309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b061ce-b40f-4fe9-988b-e2209355c15a None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:18.371635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 781/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:34:18.376942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467506-aecd-443d-abc0-34209c188501 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:18.377377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467506-aecd-443d-abc0-34209c188501 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:18.377537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467506-aecd-443d-abc0-34209c188501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:18.397495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-fe467506-aecd-443d-abc0-34209c188501 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:34:18.398725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fe467506-aecd-443d-abc0-34209c188501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:18.406648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 751/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:34:18.443065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b061ce-b40f-4fe9-988b-e2209355c15a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:34:18.453931 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b061ce-b40f-4fe9-988b-e2209355c15a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:34:18.457523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73018bd3-b1d8-4d43-b624-5943cca69c61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:34:18.458159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018bd3-b1d8-4d43-b624-5943cca69c61 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:18.458443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018bd3-b1d8-4d43-b624-5943cca69c61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:18.458725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018bd3-b1d8-4d43-b624-5943cca69c61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:18.467643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73018bd3-b1d8-4d43-b624-5943cca69c61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:34:18.469059 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73018bd3-b1d8-4d43-b624-5943cca69c61 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:34:18.478838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 782/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:34:22.483709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc5829-3ab7-4b11-9e4a-e43b47aee562 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:22.484258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc5829-3ab7-4b11-9e4a-e43b47aee562 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:22.484258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc5829-3ab7-4b11-9e4a-e43b47aee562 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:22.485675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 752/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:34:22.491603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f3641-1638-4dd0-917c-66826bf97e30 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:22.492004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f3641-1638-4dd0-917c-66826bf97e30 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:22.492162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f3641-1638-4dd0-917c-66826bf97e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:22.510233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-281f3641-1638-4dd0-917c-66826bf97e30 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:34:22.511426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-281f3641-1638-4dd0-917c-66826bf97e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:22.519250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 783/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:34:22.827039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc5829-3ab7-4b11-9e4a-e43b47aee562 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:34:22.828831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c9cc5829-3ab7-4b11-9e4a-e43b47aee562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:22.885320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc5829-3ab7-4b11-9e4a-e43b47aee562 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:34:22.888922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c3542-ba12-4045-8955-847081b4bb10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:34:22.889565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c3542-ba12-4045-8955-847081b4bb10 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:22.889868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c3542-ba12-4045-8955-847081b4bb10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:22.890129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c3542-ba12-4045-8955-847081b4bb10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:22.899144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c3542-ba12-4045-8955-847081b4bb10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:34:22.900766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c3542-ba12-4045-8955-847081b4bb10 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:34:22.910222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 753/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:34:24.970457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c36686-121c-45b1-ae8e-13ce5f91c0a0 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:24.971015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c36686-121c-45b1-ae8e-13ce5f91c0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:24.971015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c36686-121c-45b1-ae8e-13ce5f91c0a0 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:24.972438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 784/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 09:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:34:24.977575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa0b52-e766-432c-8037-bdeb8df6530b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:24.977993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa0b52-e766-432c-8037-bdeb8df6530b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:24.978137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa0b52-e766-432c-8037-bdeb8df6530b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:24.996027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ddaa0b52-e766-432c-8037-bdeb8df6530b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:34:24.997235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ddaa0b52-e766-432c-8037-bdeb8df6530b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:25.005666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 754/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 09:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:34:25.308621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c36686-121c-45b1-ae8e-13ce5f91c0a0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:34:25.310308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-32c36686-121c-45b1-ae8e-13ce5f91c0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:34:25.318487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=65 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:34:25.324124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c36686-121c-45b1-ae8e-13ce5f91c0a0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:34:25.327650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e94720-c650-431d-bfdc-fc9c12d638b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:34:25.328722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e94720-c650-431d-bfdc-fc9c12d638b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:34:25.328722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e94720-c650-431d-bfdc-fc9c12d638b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:34:25.328933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e94720-c650-431d-bfdc-fc9c12d638b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:34:25.337587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e94720-c650-431d-bfdc-fc9c12d638b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:34:25.339001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e94720-c650-431d-bfdc-fc9c12d638b7 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:34:25.348666 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 785/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:34:34.990744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=32 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:34:35.361373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:56.685573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa89e7-5a98-4df6-9f29-24fbb4361275 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:56.695681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa89e7-5a98-4df6-9f29-24fbb4361275 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:56.695681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa89e7-5a98-4df6-9f29-24fbb4361275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:56.713422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-59aa89e7-5a98-4df6-9f29-24fbb4361275 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:56.714634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-59aa89e7-5a98-4df6-9f29-24fbb4361275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:56.723129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 755/1540] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:35:56.731587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96e4a2-1251-4107-b9a1-45f5f1693aa0 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:56.732153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96e4a2-1251-4107-b9a1-45f5f1693aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:56.732153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96e4a2-1251-4107-b9a1-45f5f1693aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:56.757957 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-fd96e4a2-1251-4107-b9a1-45f5f1693aa0 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:56.759204 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fd96e4a2-1251-4107-b9a1-45f5f1693aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:56.772234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 786/1541] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:56.777645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa89e7-5a98-4df6-9f29-24fbb4361275 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:56.791608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa89e7-5a98-4df6-9f29-24fbb4361275 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:56.796852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b485e-76ff-4834-911f-97aec2d1eaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:56.797552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b485e-76ff-4834-911f-97aec2d1eaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:56.797863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b485e-76ff-4834-911f-97aec2d1eaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:56.798162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b485e-76ff-4834-911f-97aec2d1eaa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:56.798910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b485e-76ff-4834-911f-97aec2d1eaa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:35:56.799220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b485e-76ff-4834-911f-97aec2d1eaa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:56.801605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b485e-76ff-4834-911f-97aec2d1eaa7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:56.821203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 756/1542] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:35:56.829764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96e4a2-1251-4107-b9a1-45f5f1693aa0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:56.844036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96e4a2-1251-4107-b9a1-45f5f1693aa0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:56.854197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692da55b-f6d5-4b6c-a174-b65b2a4ad758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:56.855310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692da55b-f6d5-4b6c-a174-b65b2a4ad758 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:56.855854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692da55b-f6d5-4b6c-a174-b65b2a4ad758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:56.856359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692da55b-f6d5-4b6c-a174-b65b2a4ad758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:56.858035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692da55b-f6d5-4b6c-a174-b65b2a4ad758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:56.860292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692da55b-f6d5-4b6c-a174-b65b2a4ad758 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:56.943133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb5aee-d240-4289-b8f4-79800b53098f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:56.943346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb5aee-d240-4289-b8f4-79800b53098f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:56.943644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb5aee-d240-4289-b8f4-79800b53098f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:56.969602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f3bb5aee-d240-4289-b8f4-79800b53098f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:56.971129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f3bb5aee-d240-4289-b8f4-79800b53098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:56.980714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 757/1543] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:56.986261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 787/1544] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:35:56 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:56.989678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d56df-1e0e-4b70-bc8c-c9c43256c195 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:56.989972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d56df-1e0e-4b70-bc8c-c9c43256c195 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:56.990252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d56df-1e0e-4b70-bc8c-c9c43256c195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.006400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692da55b-f6d5-4b6c-a174-b65b2a4ad758 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.018909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-0f3d56df-1e0e-4b70-bc8c-c9c43256c195 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.020194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0f3d56df-1e0e-4b70-bc8c-c9c43256c195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.136666 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692da55b-f6d5-4b6c-a174-b65b2a4ad758 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.149194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed21ebd-75e1-49fa-b3f4-6b5f6db803a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.149446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 758/1545] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.150644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a60c3-b0c6-47c2-870d-280dc5c1df5f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.150944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a60c3-b0c6-47c2-870d-280dc5c1df5f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.151231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a60c3-b0c6-47c2-870d-280dc5c1df5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.151813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed21ebd-75e1-49fa-b3f4-6b5f6db803a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.152328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed21ebd-75e1-49fa-b3f4-6b5f6db803a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.152815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed21ebd-75e1-49fa-b3f4-6b5f6db803a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.153767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed21ebd-75e1-49fa-b3f4-6b5f6db803a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.155219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: WARNING py.warnings [None req-6ed21ebd-75e1-49fa-b3f4-6b5f6db803a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:35:57.155219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: warnings.warn(msg) Jun 27 09:35:57.155219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: Jun 27 09:35:57.156926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed21ebd-75e1-49fa-b3f4-6b5f6db803a2 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.194233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a44a60c3-b0c6-47c2-870d-280dc5c1df5f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.195568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a44a60c3-b0c6-47c2-870d-280dc5c1df5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.211768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 759/1546] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.212854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a60c3-b0c6-47c2-870d-280dc5c1df5f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.229784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a60c3-b0c6-47c2-870d-280dc5c1df5f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.244797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97679723-1a36-4508-8b82-393528ff147a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.246379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97679723-1a36-4508-8b82-393528ff147a admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.246883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97679723-1a36-4508-8b82-393528ff147a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.247689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97679723-1a36-4508-8b82-393528ff147a admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.248652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97679723-1a36-4508-8b82-393528ff147a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:35:57.249199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97679723-1a36-4508-8b82-393528ff147a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.251028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97679723-1a36-4508-8b82-393528ff147a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.251273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 788/1547] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:35:56 2022] POST /identity/v3/users => generated 282 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:57.252977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c297e7b-ac53-4c08-87ec-49932def5fa2 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.255327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c297e7b-ac53-4c08-87ec-49932def5fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.258074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c297e7b-ac53-4c08-87ec-49932def5fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.285237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 760/1548] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:35:57.287222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84300594-c04b-4cd2-ba16-bb6d54354026 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.287517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84300594-c04b-4cd2-ba16-bb6d54354026 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.287821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84300594-c04b-4cd2-ba16-bb6d54354026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.302569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-1c297e7b-ac53-4c08-87ec-49932def5fa2 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.303880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1c297e7b-ac53-4c08-87ec-49932def5fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.320468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-84300594-c04b-4cd2-ba16-bb6d54354026 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.324456 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-84300594-c04b-4cd2-ba16-bb6d54354026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.327470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 789/1549] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.329311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686be7c-eec6-4b29-a15e-fa2375b6683f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.329724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686be7c-eec6-4b29-a15e-fa2375b6683f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.329929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686be7c-eec6-4b29-a15e-fa2375b6683f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.338273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 761/1550] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.339639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84300594-c04b-4cd2-ba16-bb6d54354026 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.350100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84300594-c04b-4cd2-ba16-bb6d54354026 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.351179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d686be7c-eec6-4b29-a15e-fa2375b6683f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.352653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d686be7c-eec6-4b29-a15e-fa2375b6683f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.357957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6abb32-dd28-463a-9abd-761e679e1c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.358638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6abb32-dd28-463a-9abd-761e679e1c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.358917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6abb32-dd28-463a-9abd-761e679e1c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.359196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6abb32-dd28-463a-9abd-761e679e1c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.359984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6abb32-dd28-463a-9abd-761e679e1c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.361605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6abb32-dd28-463a-9abd-761e679e1c89 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.361950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 790/1551] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.363084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686be7c-eec6-4b29-a15e-fa2375b6683f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.378015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 762/1552] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:57.378235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686be7c-eec6-4b29-a15e-fa2375b6683f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.379447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365dbab-c360-4268-b157-f4f79510cba2 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.379790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365dbab-c360-4268-b157-f4f79510cba2 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.380077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365dbab-c360-4268-b157-f4f79510cba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.383949 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f353e-578d-40b1-ac50-f4d2d6febc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.384613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f353e-578d-40b1-ac50-f4d2d6febc66 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.384895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f353e-578d-40b1-ac50-f4d2d6febc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.385170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f353e-578d-40b1-ac50-f4d2d6febc66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.385863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f353e-578d-40b1-ac50-f4d2d6febc66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.387791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f353e-578d-40b1-ac50-f4d2d6febc66 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.399196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-0365dbab-c360-4268-b157-f4f79510cba2 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.400660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0365dbab-c360-4268-b157-f4f79510cba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.408678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 763/1553] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.409917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43866f-9eea-4fcc-b870-a1a25a1d62c0 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.410346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43866f-9eea-4fcc-b870-a1a25a1d62c0 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.410625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43866f-9eea-4fcc-b870-a1a25a1d62c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.428862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4c43866f-9eea-4fcc-b870-a1a25a1d62c0 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.430627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4c43866f-9eea-4fcc-b870-a1a25a1d62c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.438521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 764/1554] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.439516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459b893-c3e2-4433-b65f-15103c99d2f0 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.439828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459b893-c3e2-4433-b65f-15103c99d2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.440125 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459b893-c3e2-4433-b65f-15103c99d2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.458871 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4459b893-c3e2-4433-b65f-15103c99d2f0 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.460012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4459b893-c3e2-4433-b65f-15103c99d2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.468172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 765/1555] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.468944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459b893-c3e2-4433-b65f-15103c99d2f0 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.478243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459b893-c3e2-4433-b65f-15103c99d2f0 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.482770 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5427a-d053-47a8-b844-51fe9f6743d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.483427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5427a-d053-47a8-b844-51fe9f6743d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.483738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5427a-d053-47a8-b844-51fe9f6743d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.484009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5427a-d053-47a8-b844-51fe9f6743d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.484728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5427a-d053-47a8-b844-51fe9f6743d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:35:57.485028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5427a-d053-47a8-b844-51fe9f6743d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.486619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5427a-d053-47a8-b844-51fe9f6743d0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.498902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 766/1556] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:35:57.500166 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9982-4ccc-4a1c-a859-4ec5ae902a0c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.500452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9982-4ccc-4a1c-a859-4ec5ae902a0c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.500787 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9982-4ccc-4a1c-a859-4ec5ae902a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.519426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-370d9982-4ccc-4a1c-a859-4ec5ae902a0c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.520651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-370d9982-4ccc-4a1c-a859-4ec5ae902a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.521602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 791/1557] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:57.523082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60f2bd-eb06-4402-b1c6-61a058038c36 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.523356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60f2bd-eb06-4402-b1c6-61a058038c36 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.523665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60f2bd-eb06-4402-b1c6-61a058038c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.542978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4c60f2bd-eb06-4402-b1c6-61a058038c36 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.544225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4c60f2bd-eb06-4402-b1c6-61a058038c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.582689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 767/1558] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.583606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d9982-4ccc-4a1c-a859-4ec5ae902a0c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.589655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:57.593263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d9982-4ccc-4a1c-a859-4ec5ae902a0c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.598208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693edcc3-0413-49a8-866e-cfa81a4cc1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.601008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693edcc3-0413-49a8-866e-cfa81a4cc1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.601008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693edcc3-0413-49a8-866e-cfa81a4cc1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.601008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693edcc3-0413-49a8-866e-cfa81a4cc1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.601008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693edcc3-0413-49a8-866e-cfa81a4cc1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.601982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693edcc3-0413-49a8-866e-cfa81a4cc1e3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.605905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 792/1559] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.607260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5c54e-0ec3-4793-a8b4-eb1d1e60ff71 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.607740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5c54e-0ec3-4793-a8b4-eb1d1e60ff71 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.607920 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5c54e-0ec3-4793-a8b4-eb1d1e60ff71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.614030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 768/1560] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:57.615618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac839-d1cf-402a-8075-acb06d55fdfc None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.616103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac839-d1cf-402a-8075-acb06d55fdfc None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.616446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac839-d1cf-402a-8075-acb06d55fdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.627224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-0ce5c54e-0ec3-4793-a8b4-eb1d1e60ff71 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.628457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-0ce5c54e-0ec3-4793-a8b4-eb1d1e60ff71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.635270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-2b6ac839-d1cf-402a-8075-acb06d55fdfc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.636486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2b6ac839-d1cf-402a-8075-acb06d55fdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.638150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 793/1561] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.638770 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5c54e-0ec3-4793-a8b4-eb1d1e60ff71 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.648072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 769/1562] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.648984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ac839-d1cf-402a-8075-acb06d55fdfc None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.652142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5c54e-0ec3-4793-a8b4-eb1d1e60ff71 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.657279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.657946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.658222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.658448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ac839-d1cf-402a-8075-acb06d55fdfc None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.658573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.659203 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:35:57.659505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.660592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: WARNING py.warnings [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:35:57.660592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: warnings.warn(msg) Jun 27 09:35:57.660592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: Jun 27 09:35:57.661489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.663604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be3cc2-8281-442d-aa45-50eea5688241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.664257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be3cc2-8281-442d-aa45-50eea5688241 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.664543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be3cc2-8281-442d-aa45-50eea5688241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.664817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be3cc2-8281-442d-aa45-50eea5688241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.665510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be3cc2-8281-442d-aa45-50eea5688241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.666512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING py.warnings [None req-45be3cc2-8281-442d-aa45-50eea5688241 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:35:57.666512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: warnings.warn(msg) Jun 27 09:35:57.666512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: Jun 27 09:35:57.667726 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be3cc2-8281-442d-aa45-50eea5688241 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.674800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 794/1563] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:35:57.675667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.687737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfa489-45cf-4a1c-a5b9-76b753b4896e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.692407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae029c-979e-42c8-a979-33d0e5b056c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.693064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae029c-979e-42c8-a979-33d0e5b056c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.693350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae029c-979e-42c8-a979-33d0e5b056c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.693625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae029c-979e-42c8-a979-33d0e5b056c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.694339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae029c-979e-42c8-a979-33d0e5b056c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.696989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae029c-979e-42c8-a979-33d0e5b056c4 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.703773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:57.858891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 770/1564] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/projects => generated 367 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:57.859756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be3cc2-8281-442d-aa45-50eea5688241 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.874198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 795/1565] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/users => generated 294 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:57.874198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a96b6-d5e1-4ed5-8339-08683e1bfdee None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.874198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a96b6-d5e1-4ed5-8339-08683e1bfdee None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.874198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a96b6-d5e1-4ed5-8339-08683e1bfdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.890934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a42a96b6-d5e1-4ed5-8339-08683e1bfdee None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:57.892179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a42a96b6-d5e1-4ed5-8339-08683e1bfdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:57.931054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be3cc2-8281-442d-aa45-50eea5688241 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.936499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e47f9-185d-4837-bda4-20f83b615402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.937555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e47f9-185d-4837-bda4-20f83b615402 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.937843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e47f9-185d-4837-bda4-20f83b615402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.938480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e47f9-185d-4837-bda4-20f83b615402 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.939228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e47f9-185d-4837-bda4-20f83b615402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:35:57.939550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e47f9-185d-4837-bda4-20f83b615402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.941225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e47f9-185d-4837-bda4-20f83b615402 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.953308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 771/1566] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:35:57.954276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e47f9-185d-4837-bda4-20f83b615402 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.957561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 796/1567] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:57.958572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a96b6-d5e1-4ed5-8339-08683e1bfdee None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:57.966117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e47f9-185d-4837-bda4-20f83b615402 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.970902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24d802-1a54-4c04-a9ee-5460e62e329e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.971854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24d802-1a54-4c04-a9ee-5460e62e329e admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.972049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a96b6-d5e1-4ed5-8339-08683e1bfdee None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:57.972185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24d802-1a54-4c04-a9ee-5460e62e329e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.972405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24d802-1a54-4c04-a9ee-5460e62e329e admin admin] PATH_INFO: `/v3/projects/f4e9b909eb8244d3a025239ee1e8448e/users/6ae4a199cf0f41a38eb6d4c283beebc5/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.978091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abf36b-1373-4e7a-9089-d176c0b3dbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:57.978741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abf36b-1373-4e7a-9089-d176c0b3dbff admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:57.979017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abf36b-1373-4e7a-9089-d176c0b3dbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:57.979289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abf36b-1373-4e7a-9089-d176c0b3dbff admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:57.980050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abf36b-1373-4e7a-9089-d176c0b3dbff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:35:57.980355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abf36b-1373-4e7a-9089-d176c0b3dbff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.982720 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5abf36b-1373-4e7a-9089-d176c0b3dbff admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.988123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24d802-1a54-4c04-a9ee-5460e62e329e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e9b909eb8244d3a025239ee1e8448e, user_id=6ae4a199cf0f41a38eb6d4c283beebc5, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:57.989181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING py.warnings [None req-6d24d802-1a54-4c04-a9ee-5460e62e329e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:35:57.989181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: warnings.warn(msg) Jun 27 09:35:57.989181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: Jun 27 09:35:57.991130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24d802-1a54-4c04-a9ee-5460e62e329e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:57.995989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 797/1568] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:35:57.996801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abf36b-1373-4e7a-9089-d176c0b3dbff admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.006721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5abf36b-1373-4e7a-9089-d176c0b3dbff admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.017439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472c331-aa1a-4e86-bc17-ac9823c09ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.017439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472c331-aa1a-4e86-bc17-ac9823c09ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.017439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472c331-aa1a-4e86-bc17-ac9823c09ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.017439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472c331-aa1a-4e86-bc17-ac9823c09ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.017818 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472c331-aa1a-4e86-bc17-ac9823c09ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:35:58.017818 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472c331-aa1a-4e86-bc17-ac9823c09ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.017818 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472c331-aa1a-4e86-bc17-ac9823c09ee3 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.027709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 798/1569] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:35:58.028502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472c331-aa1a-4e86-bc17-ac9823c09ee3 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.039208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472c331-aa1a-4e86-bc17-ac9823c09ee3 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.043861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6871f8d-aa22-4a29-83ba-f5e9a7b71396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.044551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6871f8d-aa22-4a29-83ba-f5e9a7b71396 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.044833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6871f8d-aa22-4a29-83ba-f5e9a7b71396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.045110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6871f8d-aa22-4a29-83ba-f5e9a7b71396 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.045796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6871f8d-aa22-4a29-83ba-f5e9a7b71396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.047873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6871f8d-aa22-4a29-83ba-f5e9a7b71396 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.052583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 772/1570] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:35:57 2022] PUT /identity/v3/projects/f4e9b909eb8244d3a025239ee1e8448e/users/6ae4a199cf0f41a38eb6d4c283beebc5/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:35:58.053231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24d802-1a54-4c04-a9ee-5460e62e329e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.125157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24d802-1a54-4c04-a9ee-5460e62e329e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.128079 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 799/1571] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.128824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6871f8d-aa22-4a29-83ba-f5e9a7b71396 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.133109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c03c23-efdb-40f7-82bb-093bc1bc3061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.133777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c03c23-efdb-40f7-82bb-093bc1bc3061 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.134060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c03c23-efdb-40f7-82bb-093bc1bc3061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.134334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c03c23-efdb-40f7-82bb-093bc1bc3061 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.135039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c03c23-efdb-40f7-82bb-093bc1bc3061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.137011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c03c23-efdb-40f7-82bb-093bc1bc3061 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.195113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6871f8d-aa22-4a29-83ba-f5e9a7b71396 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.199782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd8bc6-0863-4215-ae62-92de4d2ce346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.200457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd8bc6-0863-4215-ae62-92de4d2ce346 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.200734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd8bc6-0863-4215-ae62-92de4d2ce346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.201007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd8bc6-0863-4215-ae62-92de4d2ce346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.201725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd8bc6-0863-4215-ae62-92de4d2ce346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.203324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd8bc6-0863-4215-ae62-92de4d2ce346 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.213760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 800/1572] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:58.214524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd8bc6-0863-4215-ae62-92de4d2ce346 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.223890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd8bc6-0863-4215-ae62-92de4d2ce346 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.227683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 773/1573] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.227683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c03c23-efdb-40f7-82bb-093bc1bc3061 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.229715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7b34c-b80c-4320-b1e4-1680af1fbc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.230369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7b34c-b80c-4320-b1e4-1680af1fbc29 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.230662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7b34c-b80c-4320-b1e4-1680af1fbc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.230941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7b34c-b80c-4320-b1e4-1680af1fbc29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.231718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7b34c-b80c-4320-b1e4-1680af1fbc29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.233935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7b34c-b80c-4320-b1e4-1680af1fbc29 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.236672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c03c23-efdb-40f7-82bb-093bc1bc3061 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.241771 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b3a39-b174-4bce-ab65-40572436bb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.242444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b3a39-b174-4bce-ab65-40572436bb57 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.242731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b3a39-b174-4bce-ab65-40572436bb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.243015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b3a39-b174-4bce-ab65-40572436bb57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.243903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b3a39-b174-4bce-ab65-40572436bb57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:35:58.244216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b3a39-b174-4bce-ab65-40572436bb57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.245819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b3a39-b174-4bce-ab65-40572436bb57 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.258485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 774/1574] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:35:58.259469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b3a39-b174-4bce-ab65-40572436bb57 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.269197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b3a39-b174-4bce-ab65-40572436bb57 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.274903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e518bee-b812-4d78-90eb-a2cdbf463736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.275689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518bee-b812-4d78-90eb-a2cdbf463736 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.275879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518bee-b812-4d78-90eb-a2cdbf463736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.276162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518bee-b812-4d78-90eb-a2cdbf463736 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.276978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e518bee-b812-4d78-90eb-a2cdbf463736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.278966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e518bee-b812-4d78-90eb-a2cdbf463736 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.287396 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:58.309929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 801/1575] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/projects => generated 393 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.310750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7b34c-b80c-4320-b1e4-1680af1fbc29 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.331725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:58.375905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7b34c-b80c-4320-b1e4-1680af1fbc29 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.380478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb80cfa-1ede-43c2-8a8d-66f016c685cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.381218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb80cfa-1ede-43c2-8a8d-66f016c685cd admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.381506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb80cfa-1ede-43c2-8a8d-66f016c685cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.381783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb80cfa-1ede-43c2-8a8d-66f016c685cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.382475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb80cfa-1ede-43c2-8a8d-66f016c685cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.384410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb80cfa-1ede-43c2-8a8d-66f016c685cd admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.407261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 775/1576] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.412595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b681-bc95-4606-acca-ba66102a4993 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.412595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b681-bc95-4606-acca-ba66102a4993 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.412595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b681-bc95-4606-acca-ba66102a4993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.436294 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-5b71b681-bc95-4606-acca-ba66102a4993 None None] MFA Rules not processed for user `6ae4a199cf0f41a38eb6d4c283beebc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:58.437511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5b71b681-bc95-4606-acca-ba66102a4993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:58.469810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 802/1577] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.470903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb80cfa-1ede-43c2-8a8d-66f016c685cd admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.504732 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 776/1578] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:58.505958 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b681-bc95-4606-acca-ba66102a4993 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.537095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb80cfa-1ede-43c2-8a8d-66f016c685cd admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.542727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27036d33-361e-47bc-94cd-4f66f9fdce22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.543075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27036d33-361e-47bc-94cd-4f66f9fdce22 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.544093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27036d33-361e-47bc-94cd-4f66f9fdce22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.544373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27036d33-361e-47bc-94cd-4f66f9fdce22 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.545068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27036d33-361e-47bc-94cd-4f66f9fdce22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.547012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27036d33-361e-47bc-94cd-4f66f9fdce22 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.569344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b681-bc95-4606-acca-ba66102a4993 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.574343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134a58-c6e3-41df-bbc9-b69a9acd8bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.575005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134a58-c6e3-41df-bbc9-b69a9acd8bdb admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.575279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134a58-c6e3-41df-bbc9-b69a9acd8bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.575573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29134a58-c6e3-41df-bbc9-b69a9acd8bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.576296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29134a58-c6e3-41df-bbc9-b69a9acd8bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.577903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29134a58-c6e3-41df-bbc9-b69a9acd8bdb admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.589026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 777/1579] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:58.589850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134a58-c6e3-41df-bbc9-b69a9acd8bdb admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.599147 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29134a58-c6e3-41df-bbc9-b69a9acd8bdb admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.603952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b59ef-cb79-4c21-894b-ce2e4de3f704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.604576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b59ef-cb79-4c21-894b-ce2e4de3f704 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.604852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b59ef-cb79-4c21-894b-ce2e4de3f704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.605122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b59ef-cb79-4c21-894b-ce2e4de3f704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.605846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b59ef-cb79-4c21-894b-ce2e4de3f704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.607403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b59ef-cb79-4c21-894b-ce2e4de3f704 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.617040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 778/1580] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:58.617785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b59ef-cb79-4c21-894b-ce2e4de3f704 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.630795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b59ef-cb79-4c21-894b-ce2e4de3f704 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.631630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f84b4-0d7e-4716-8876-4e42d657179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.632769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f84b4-0d7e-4716-8876-4e42d657179c admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.633050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f84b4-0d7e-4716-8876-4e42d657179c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.633324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f84b4-0d7e-4716-8876-4e42d657179c admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.633989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f84b4-0d7e-4716-8876-4e42d657179c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.635896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f84b4-0d7e-4716-8876-4e42d657179c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.656000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 803/1581] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/users => generated 303 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.656000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27036d33-361e-47bc-94cd-4f66f9fdce22 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.663744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27036d33-361e-47bc-94cd-4f66f9fdce22 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.668289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b88ff-0886-4d6c-96f7-403ad90e3871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.671072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b88ff-0886-4d6c-96f7-403ad90e3871 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.671072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b88ff-0886-4d6c-96f7-403ad90e3871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.671072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b88ff-0886-4d6c-96f7-403ad90e3871 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.671072 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b88ff-0886-4d6c-96f7-403ad90e3871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.672092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b88ff-0886-4d6c-96f7-403ad90e3871 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.710046 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 779/1582] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.710913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f84b4-0d7e-4716-8876-4e42d657179c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.732596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 804/1583] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/users => generated 297 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.732596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b88ff-0886-4d6c-96f7-403ad90e3871 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.743073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b88ff-0886-4d6c-96f7-403ad90e3871 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.757629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:58.774663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f84b4-0d7e-4716-8876-4e42d657179c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.780035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739eb79-1da8-4f9a-9d4e-a5ef3ae13b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.780704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739eb79-1da8-4f9a-9d4e-a5ef3ae13b1b admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.780996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739eb79-1da8-4f9a-9d4e-a5ef3ae13b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.781253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739eb79-1da8-4f9a-9d4e-a5ef3ae13b1b admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.781939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739eb79-1da8-4f9a-9d4e-a5ef3ae13b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.783893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739eb79-1da8-4f9a-9d4e-a5ef3ae13b1b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.802246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e7ccd-6740-4d65-badf-1a63dded4488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.803101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e7ccd-6740-4d65-badf-1a63dded4488 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.803285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e7ccd-6740-4d65-badf-1a63dded4488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.803588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e7ccd-6740-4d65-badf-1a63dded4488 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.804338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e7ccd-6740-4d65-badf-1a63dded4488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.806302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e7ccd-6740-4d65-badf-1a63dded4488 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.917365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 805/1585] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.918260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 780/1585] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/users => generated 283 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:58.918260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739eb79-1da8-4f9a-9d4e-a5ef3ae13b1b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.918718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e7ccd-6740-4d65-badf-1a63dded4488 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:58.920336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:58.926334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e7ccd-6740-4d65-badf-1a63dded4488 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.926988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739eb79-1da8-4f9a-9d4e-a5ef3ae13b1b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:58.932005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d4453-b07f-4b82-9bea-f3b81e090568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.932222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3753ad6-2033-4a2d-9bd4-94bd6cbbd0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:58.932705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d4453-b07f-4b82-9bea-f3b81e090568 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.932993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d4453-b07f-4b82-9bea-f3b81e090568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.933116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3753ad6-2033-4a2d-9bd4-94bd6cbbd0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:58.933270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d4453-b07f-4b82-9bea-f3b81e090568 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.933367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3753ad6-2033-4a2d-9bd4-94bd6cbbd0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:58.933626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3753ad6-2033-4a2d-9bd4-94bd6cbbd0f1 admin admin] PATH_INFO: `/v3/projects/79203b0660c041f49c5f2629bd5a0bdf/users/ecf085030e8e4a65a7140fea2b64a7ed/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:58.933967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d4453-b07f-4b82-9bea-f3b81e090568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.936036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d4453-b07f-4b82-9bea-f3b81e090568 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:58.950605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3753ad6-2033-4a2d-9bd4-94bd6cbbd0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79203b0660c041f49c5f2629bd5a0bdf, user_id=ecf085030e8e4a65a7140fea2b64a7ed, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:58.953488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3753ad6-2033-4a2d-9bd4-94bd6cbbd0f1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.001193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 806/1586] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:35:58 2022] POST /identity/v3/projects => generated 365 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:59.002028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d4453-b07f-4b82-9bea-f3b81e090568 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.017091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 781/1587] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:35:58 2022] PUT /identity/v3/projects/79203b0660c041f49c5f2629bd5a0bdf/users/ecf085030e8e4a65a7140fea2b64a7ed/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:35:59.019239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3753ad6-2033-4a2d-9bd4-94bd6cbbd0f1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.066512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d4453-b07f-4b82-9bea-f3b81e090568 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.080917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3753ad6-2033-4a2d-9bd4-94bd6cbbd0f1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.085603 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2611824-8d0b-4bd2-bb6c-7d1da92c94ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.086244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2611824-8d0b-4bd2-bb6c-7d1da92c94ec admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.086530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2611824-8d0b-4bd2-bb6c-7d1da92c94ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.086809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2611824-8d0b-4bd2-bb6c-7d1da92c94ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.087559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2611824-8d0b-4bd2-bb6c-7d1da92c94ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.089165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2611824-8d0b-4bd2-bb6c-7d1da92c94ec admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.099019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 782/1588] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.102259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2611824-8d0b-4bd2-bb6c-7d1da92c94ec admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.109299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2611824-8d0b-4bd2-bb6c-7d1da92c94ec admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.125525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01a5f5-7571-4735-81dc-f3df688b7fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.126231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01a5f5-7571-4735-81dc-f3df688b7fab admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.126522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01a5f5-7571-4735-81dc-f3df688b7fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.126798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01a5f5-7571-4735-81dc-f3df688b7fab admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.127504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01a5f5-7571-4735-81dc-f3df688b7fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.129484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01a5f5-7571-4735-81dc-f3df688b7fab admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.131438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc002a6-cec4-4b70-aea8-df63477821f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.132144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc002a6-cec4-4b70-aea8-df63477821f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.132425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc002a6-cec4-4b70-aea8-df63477821f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.132699 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc002a6-cec4-4b70-aea8-df63477821f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.134046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc002a6-cec4-4b70-aea8-df63477821f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.135659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc002a6-cec4-4b70-aea8-df63477821f2 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.146208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 807/1589] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.147043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc002a6-cec4-4b70-aea8-df63477821f2 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.157215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc002a6-cec4-4b70-aea8-df63477821f2 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.162013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74286d96-3f46-42df-a6f7-bbaff05b4af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.162698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74286d96-3f46-42df-a6f7-bbaff05b4af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.162985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74286d96-3f46-42df-a6f7-bbaff05b4af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.163513 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74286d96-3f46-42df-a6f7-bbaff05b4af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.164321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74286d96-3f46-42df-a6f7-bbaff05b4af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.165884 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74286d96-3f46-42df-a6f7-bbaff05b4af1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.176020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 808/1590] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.176753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74286d96-3f46-42df-a6f7-bbaff05b4af1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.186742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74286d96-3f46-42df-a6f7-bbaff05b4af1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.191468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c557ce-2692-4b1c-b551-5ea63dc72d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.193910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c557ce-2692-4b1c-b551-5ea63dc72d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.193910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c557ce-2692-4b1c-b551-5ea63dc72d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.193910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c557ce-2692-4b1c-b551-5ea63dc72d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.193910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c557ce-2692-4b1c-b551-5ea63dc72d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.195004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c557ce-2692-4b1c-b551-5ea63dc72d3d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.205986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 809/1591] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.206782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c557ce-2692-4b1c-b551-5ea63dc72d3d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.216503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c557ce-2692-4b1c-b551-5ea63dc72d3d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.221250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9224a-eea3-4dbf-91ce-b5811264082d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.221909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9224a-eea3-4dbf-91ce-b5811264082d admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.222272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9224a-eea3-4dbf-91ce-b5811264082d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.222631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9224a-eea3-4dbf-91ce-b5811264082d admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.224373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 783/1592] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:35:59 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:59.224504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9224a-eea3-4dbf-91ce-b5811264082d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.224589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01a5f5-7571-4735-81dc-f3df688b7fab admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.225191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9224a-eea3-4dbf-91ce-b5811264082d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.234082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01a5f5-7571-4735-81dc-f3df688b7fab admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.240158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 810/1593] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.240308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942fb7b-ff66-4ce0-af0d-063522ef6a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.240411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae76ab-93ce-4b06-8633-b9b9405db851 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.240411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae76ab-93ce-4b06-8633-b9b9405db851 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.240411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae76ab-93ce-4b06-8633-b9b9405db851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.240660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942fb7b-ff66-4ce0-af0d-063522ef6a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.240660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942fb7b-ff66-4ce0-af0d-063522ef6a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.240660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942fb7b-ff66-4ce0-af0d-063522ef6a5c admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.240660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942fb7b-ff66-4ce0-af0d-063522ef6a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.242927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942fb7b-ff66-4ce0-af0d-063522ef6a5c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.257261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-abae76ab-93ce-4b06-8633-b9b9405db851 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:59.258476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-abae76ab-93ce-4b06-8633-b9b9405db851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:59.266183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 811/1594] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:59.267119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae76ab-93ce-4b06-8633-b9b9405db851 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.277887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae76ab-93ce-4b06-8633-b9b9405db851 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.282108 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f7b6f-a2b0-4f32-b7c1-e0b099792a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.282777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f7b6f-a2b0-4f32-b7c1-e0b099792a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.283053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f7b6f-a2b0-4f32-b7c1-e0b099792a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.283326 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f7b6f-a2b0-4f32-b7c1-e0b099792a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.284202 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f7b6f-a2b0-4f32-b7c1-e0b099792a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.285842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f7b6f-a2b0-4f32-b7c1-e0b099792a41 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.296910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 812/1595] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.297702 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f7b6f-a2b0-4f32-b7c1-e0b099792a41 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.308225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f7b6f-a2b0-4f32-b7c1-e0b099792a41 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.312891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75002c74-e76a-4403-9683-1b9489eca0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.313776 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75002c74-e76a-4403-9683-1b9489eca0ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.314060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75002c74-e76a-4403-9683-1b9489eca0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.314365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75002c74-e76a-4403-9683-1b9489eca0ef admin admin] PATH_INFO: `/v3/projects/e6d5085e18534f11823c5d1f9375caf4/users/a07977e47fdc4bb0acdcc82e8fa694dd/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.330389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75002c74-e76a-4403-9683-1b9489eca0ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d5085e18534f11823c5d1f9375caf4, user_id=a07977e47fdc4bb0acdcc82e8fa694dd, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.331911 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 784/1596] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:35:59 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:35:59.332740 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942fb7b-ff66-4ce0-af0d-063522ef6a5c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.333093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75002c74-e76a-4403-9683-1b9489eca0ef admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.343956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942fb7b-ff66-4ce0-af0d-063522ef6a5c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.349355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0bd38-de3a-456e-95b7-24fe323e1c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.350022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0bd38-de3a-456e-95b7-24fe323e1c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.350307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0bd38-de3a-456e-95b7-24fe323e1c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.350581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0bd38-de3a-456e-95b7-24fe323e1c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.351299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0bd38-de3a-456e-95b7-24fe323e1c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.353466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0bd38-de3a-456e-95b7-24fe323e1c50 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.364213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 785/1597] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.365031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0bd38-de3a-456e-95b7-24fe323e1c50 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.375017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0bd38-de3a-456e-95b7-24fe323e1c50 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.379682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfc561-4c3a-45f1-870d-5de7cb6c1033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.380327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc561-4c3a-45f1-870d-5de7cb6c1033 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.380610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc561-4c3a-45f1-870d-5de7cb6c1033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.380887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc561-4c3a-45f1-870d-5de7cb6c1033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.381600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfc561-4c3a-45f1-870d-5de7cb6c1033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.383209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfc561-4c3a-45f1-870d-5de7cb6c1033 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.392911 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 786/1598] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.393679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfc561-4c3a-45f1-870d-5de7cb6c1033 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.402824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfc561-4c3a-45f1-870d-5de7cb6c1033 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.407666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04563e-fd25-41d4-b4c4-c4c45aa4a0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.408283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04563e-fd25-41d4-b4c4-c4c45aa4a0ea admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.408555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04563e-fd25-41d4-b4c4-c4c45aa4a0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.408830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04563e-fd25-41d4-b4c4-c4c45aa4a0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.409541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04563e-fd25-41d4-b4c4-c4c45aa4a0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.411054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04563e-fd25-41d4-b4c4-c4c45aa4a0ea admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.421705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 787/1599] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.424799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04563e-fd25-41d4-b4c4-c4c45aa4a0ea admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.424909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 813/1600] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:35:59 2022] PUT /identity/v3/projects/e6d5085e18534f11823c5d1f9375caf4/users/a07977e47fdc4bb0acdcc82e8fa694dd/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:35:59.424909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75002c74-e76a-4403-9683-1b9489eca0ef admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.435147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:59.441382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04563e-fd25-41d4-b4c4-c4c45aa4a0ea admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.472916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:59.479224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75002c74-e76a-4403-9683-1b9489eca0ef admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.483188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acabe67-0528-41c9-accf-4b837e48c1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.483945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acabe67-0528-41c9-accf-4b837e48c1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.484229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acabe67-0528-41c9-accf-4b837e48c1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.484504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acabe67-0528-41c9-accf-4b837e48c1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.498170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75f4af-f162-4b6d-9f7c-ace265cde7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.498884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75f4af-f162-4b6d-9f7c-ace265cde7ec admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.499167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75f4af-f162-4b6d-9f7c-ace265cde7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.499441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75f4af-f162-4b6d-9f7c-ace265cde7ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.500211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75f4af-f162-4b6d-9f7c-ace265cde7ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.501970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75f4af-f162-4b6d-9f7c-ace265cde7ec admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.512587 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 788/1601] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.513524 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75f4af-f162-4b6d-9f7c-ace265cde7ec admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.523090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75f4af-f162-4b6d-9f7c-ace265cde7ec admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.527946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d91a7-d336-47bf-883b-e7887723754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.528909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d91a7-d336-47bf-883b-e7887723754e admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.529375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d91a7-d336-47bf-883b-e7887723754e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.530073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d91a7-d336-47bf-883b-e7887723754e admin admin] PATH_INFO: `/v3/projects/e93273a70b35491d8d0acf14ddb5afe4/users/76bf077a5259406f81aff8af3b2b3a02/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.545620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d91a7-d336-47bf-883b-e7887723754e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93273a70b35491d8d0acf14ddb5afe4, user_id=76bf077a5259406f81aff8af3b2b3a02, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.545952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acabe67-0528-41c9-accf-4b837e48c1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.547423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acabe67-0528-41c9-accf-4b837e48c1a3 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.548325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d91a7-d336-47bf-883b-e7887723754e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.558343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 814/1602] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:35:59.559147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acabe67-0528-41c9-accf-4b837e48c1a3 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.568368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acabe67-0528-41c9-accf-4b837e48c1a3 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.573028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c50cc-7e06-488c-a2e2-2d4ff7f1fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.573686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c50cc-7e06-488c-a2e2-2d4ff7f1fa21 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.573963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c50cc-7e06-488c-a2e2-2d4ff7f1fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.574234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c50cc-7e06-488c-a2e2-2d4ff7f1fa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.574964 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c50cc-7e06-488c-a2e2-2d4ff7f1fa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.576598 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c50cc-7e06-488c-a2e2-2d4ff7f1fa21 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.587809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 815/1603] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.588812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c50cc-7e06-488c-a2e2-2d4ff7f1fa21 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.598443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c50cc-7e06-488c-a2e2-2d4ff7f1fa21 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.606870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 789/1604] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:35:59 2022] PUT /identity/v3/projects/e93273a70b35491d8d0acf14ddb5afe4/users/76bf077a5259406f81aff8af3b2b3a02/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:35:59.607735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d91a7-d336-47bf-883b-e7887723754e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.657322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91be568-8f6c-4c6f-8988-6e9706b47424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.658221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91be568-8f6c-4c6f-8988-6e9706b47424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.658500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91be568-8f6c-4c6f-8988-6e9706b47424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.658773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91be568-8f6c-4c6f-8988-6e9706b47424 admin admin] PATH_INFO: `/v3/projects/6a50c1d09d3247728aa1ef9fb69463fc/users/41ce5cc7ff504d64aac73e48097a4b01/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.672846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d91a7-d336-47bf-883b-e7887723754e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.674803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91be568-8f6c-4c6f-8988-6e9706b47424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a50c1d09d3247728aa1ef9fb69463fc, user_id=41ce5cc7ff504d64aac73e48097a4b01, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.677676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91be568-8f6c-4c6f-8988-6e9706b47424 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.677858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ef721-c3aa-4352-8025-53ec5d5309d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.678663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ef721-c3aa-4352-8025-53ec5d5309d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.678945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ef721-c3aa-4352-8025-53ec5d5309d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.679216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ef721-c3aa-4352-8025-53ec5d5309d5 admin admin] PATH_INFO: `/v3/projects/72b88c3f520e421bbfabac2620d19f0b/users/d11f89ecadc5445ca2b92bf4592f1b04/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.695321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ef721-c3aa-4352-8025-53ec5d5309d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b88c3f520e421bbfabac2620d19f0b, user_id=d11f89ecadc5445ca2b92bf4592f1b04, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.698559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ef721-c3aa-4352-8025-53ec5d5309d5 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.766499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 816/1605] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:35:59 2022] PUT /identity/v3/projects/6a50c1d09d3247728aa1ef9fb69463fc/users/41ce5cc7ff504d64aac73e48097a4b01/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:35:59.767378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91be568-8f6c-4c6f-8988-6e9706b47424 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.800341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:35:59.824051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 790/1606] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:35:59 2022] PUT /identity/v3/projects/72b88c3f520e421bbfabac2620d19f0b/users/d11f89ecadc5445ca2b92bf4592f1b04/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:35:59.825905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41edd6b-88a7-40be-83b6-1bbdd535491b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.825905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41edd6b-88a7-40be-83b6-1bbdd535491b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.831597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41edd6b-88a7-40be-83b6-1bbdd535491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.831697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91be568-8f6c-4c6f-8988-6e9706b47424 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.858583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e41edd6b-88a7-40be-83b6-1bbdd535491b None None] MFA Rules not processed for user `a07977e47fdc4bb0acdcc82e8fa694dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:35:59.861004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e41edd6b-88a7-40be-83b6-1bbdd535491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:35:59.896364 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a37a23-1831-40da-b837-a06c53018add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.897057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37a23-1831-40da-b837-a06c53018add admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.897343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37a23-1831-40da-b837-a06c53018add admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.897619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37a23-1831-40da-b837-a06c53018add admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.898397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a37a23-1831-40da-b837-a06c53018add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.900302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a37a23-1831-40da-b837-a06c53018add admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.912436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 817/1607] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:35:59.915907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a37a23-1831-40da-b837-a06c53018add admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.926570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a37a23-1831-40da-b837-a06c53018add admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.933244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc2e2e-6b0f-4a27-90d2-bde8711f28b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.934148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc2e2e-6b0f-4a27-90d2-bde8711f28b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.936578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc2e2e-6b0f-4a27-90d2-bde8711f28b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.936578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc2e2e-6b0f-4a27-90d2-bde8711f28b2 admin admin] PATH_INFO: `/v3/projects/eb3c0fa49ea5424399304b9f9b18e866/users/0e7a041f66e245189135fe8bedc93448/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.958081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 791/1608] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:35:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:35:59.959011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc2e2e-6b0f-4a27-90d2-bde8711f28b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3c0fa49ea5424399304b9f9b18e866, user_id=0e7a041f66e245189135fe8bedc93448, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.959133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41edd6b-88a7-40be-83b6-1bbdd535491b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:35:59.961861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc2e2e-6b0f-4a27-90d2-bde8711f28b2 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.971127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41edd6b-88a7-40be-83b6-1bbdd535491b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:35:59.976200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6e13d-a21b-45e0-9dfc-f4e533511d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:35:59.977618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6e13d-a21b-45e0-9dfc-f4e533511d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:35:59.977898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6e13d-a21b-45e0-9dfc-f4e533511d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:35:59.978173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6e13d-a21b-45e0-9dfc-f4e533511d62 admin admin] PATH_INFO: `/v3/projects/02636e4224fc4543bc39f718a57aa73c/users/55fd6ab0b4ae4661b0b86133c5fb689a/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:35:59.993718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6e13d-a21b-45e0-9dfc-f4e533511d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02636e4224fc4543bc39f718a57aa73c, user_id=55fd6ab0b4ae4661b0b86133c5fb689a, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:35:59.996436 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6e13d-a21b-45e0-9dfc-f4e533511d62 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:35:59.999542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:00.059092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 818/1609] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:35:59 2022] PUT /identity/v3/projects/eb3c0fa49ea5424399304b9f9b18e866/users/0e7a041f66e245189135fe8bedc93448/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:00.059092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d38cb-8778-4d72-b787-6a18fc0e104b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.059092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d38cb-8778-4d72-b787-6a18fc0e104b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.059092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d38cb-8778-4d72-b787-6a18fc0e104b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.072393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-395d38cb-8778-4d72-b787-6a18fc0e104b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:00.073707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-395d38cb-8778-4d72-b787-6a18fc0e104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:00.099956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 792/1610] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:35:59 2022] PUT /identity/v3/projects/02636e4224fc4543bc39f718a57aa73c/users/55fd6ab0b4ae4661b0b86133c5fb689a/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:00.101289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a26c46-bb92-481f-aa33-15ee41d3cb24 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.102119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a26c46-bb92-481f-aa33-15ee41d3cb24 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.102401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a26c46-bb92-481f-aa33-15ee41d3cb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.130265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-21a26c46-bb92-481f-aa33-15ee41d3cb24 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:00.130265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-21a26c46-bb92-481f-aa33-15ee41d3cb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:00.142784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 819/1611] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:00.144171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed2e63-6b26-45a0-9e9e-05bbdea88b19 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.144466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed2e63-6b26-45a0-9e9e-05bbdea88b19 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.144819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed2e63-6b26-45a0-9e9e-05bbdea88b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.177559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-71ed2e63-6b26-45a0-9e9e-05bbdea88b19 None None] MFA Rules not processed for user `d11f89ecadc5445ca2b92bf4592f1b04`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:00.178948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-71ed2e63-6b26-45a0-9e9e-05bbdea88b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:00.218942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 793/1612] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:00.220269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a26c46-bb92-481f-aa33-15ee41d3cb24 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:00.231884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a26c46-bb92-481f-aa33-15ee41d3cb24 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:00.235215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:00.236957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae5208-fcbd-4903-9e56-4547f444b631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:00.237600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5208-fcbd-4903-9e56-4547f444b631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.237872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5208-fcbd-4903-9e56-4547f444b631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.238175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5208-fcbd-4903-9e56-4547f444b631 admin admin] PATH_INFO: `/v3/projects/84dd698260134176acd4f4892911ab05/users/9b5d5a3f43c544368118125f2e194565/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.254615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae5208-fcbd-4903-9e56-4547f444b631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84dd698260134176acd4f4892911ab05, user_id=9b5d5a3f43c544368118125f2e194565, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:00.257410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae5208-fcbd-4903-9e56-4547f444b631 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:00.261094 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 820/1613] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:00.261943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed2e63-6b26-45a0-9e9e-05bbdea88b19 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:00.317142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed2e63-6b26-45a0-9e9e-05bbdea88b19 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:00.320792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b217854-4485-49d6-9ab0-f09119c1523d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:00.321439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b217854-4485-49d6-9ab0-f09119c1523d service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.321722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b217854-4485-49d6-9ab0-f09119c1523d service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.321999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b217854-4485-49d6-9ab0-f09119c1523d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.377397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 794/1614] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:00 2022] PUT /identity/v3/projects/84dd698260134176acd4f4892911ab05/users/9b5d5a3f43c544368118125f2e194565/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:00.379062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844657-5ee1-4515-82ea-3af3c2a6ba80 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.379559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844657-5ee1-4515-82ea-3af3c2a6ba80 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.380013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad844657-5ee1-4515-82ea-3af3c2a6ba80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.390856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b217854-4485-49d6-9ab0-f09119c1523d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:00.392610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b217854-4485-49d6-9ab0-f09119c1523d service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:00.405804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 821/1615] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:00.407142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b217854-4485-49d6-9ab0-f09119c1523d service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:00.423194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b217854-4485-49d6-9ab0-f09119c1523d service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:00.426290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ad844657-5ee1-4515-82ea-3af3c2a6ba80 None None] MFA Rules not processed for user `0e7a041f66e245189135fe8bedc93448`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:00.427904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ad844657-5ee1-4515-82ea-3af3c2a6ba80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:00.499614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc2f9e-1aaa-410d-9974-301d6dda75fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:00.500441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc2f9e-1aaa-410d-9974-301d6dda75fc admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.503242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc2f9e-1aaa-410d-9974-301d6dda75fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.503242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc2f9e-1aaa-410d-9974-301d6dda75fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.503242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc2f9e-1aaa-410d-9974-301d6dda75fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:00.503753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc2f9e-1aaa-410d-9974-301d6dda75fc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:00.508232 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 795/1616] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:00.509123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad844657-5ee1-4515-82ea-3af3c2a6ba80 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:00.534390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 822/1617] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:00.534390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc2f9e-1aaa-410d-9974-301d6dda75fc admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:00.576568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:00.576764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad844657-5ee1-4515-82ea-3af3c2a6ba80 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:00.580867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8339b9-3107-4f54-a053-9fd801790460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:00.581689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8339b9-3107-4f54-a053-9fd801790460 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.581976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8339b9-3107-4f54-a053-9fd801790460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.582251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8339b9-3107-4f54-a053-9fd801790460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.593843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8339b9-3107-4f54-a053-9fd801790460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:00.598013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8339b9-3107-4f54-a053-9fd801790460 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:00.608524 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 796/1618] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:00.609410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8339b9-3107-4f54-a053-9fd801790460 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:00.613516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc2f9e-1aaa-410d-9974-301d6dda75fc admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:00.616220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:00.620368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4ec2d-d0c5-41a1-a5ee-7c0a2e2f2505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:00.620724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8339b9-3107-4f54-a053-9fd801790460 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:00.621470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4ec2d-d0c5-41a1-a5ee-7c0a2e2f2505 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.621941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4ec2d-d0c5-41a1-a5ee-7c0a2e2f2505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.622352 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4ec2d-d0c5-41a1-a5ee-7c0a2e2f2505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.625932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bfbce-53bb-4e12-909a-d9357f9682a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:00.630777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bfbce-53bb-4e12-909a-d9357f9682a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.630777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bfbce-53bb-4e12-909a-d9357f9682a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.630777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bfbce-53bb-4e12-909a-d9357f9682a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.630777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bfbce-53bb-4e12-909a-d9357f9682a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:00.630777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bfbce-53bb-4e12-909a-d9357f9682a0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:00.634535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4ec2d-d0c5-41a1-a5ee-7c0a2e2f2505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:00.636765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4ec2d-d0c5-41a1-a5ee-7c0a2e2f2505 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:00.653745 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 823/1619] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:00.656149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782971-dfe0-4ad0-a808-4d3738fda37f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.656561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782971-dfe0-4ad0-a808-4d3738fda37f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.657093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782971-dfe0-4ad0-a808-4d3738fda37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.678406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c9782971-dfe0-4ad0-a808-4d3738fda37f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:00.679875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c9782971-dfe0-4ad0-a808-4d3738fda37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:00.688295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 824/1620] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:00.689343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782971-dfe0-4ad0-a808-4d3738fda37f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:00.700258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782971-dfe0-4ad0-a808-4d3738fda37f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:00.705575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9022a1-e78e-4536-959e-f250cd5fc883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:00.706763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9022a1-e78e-4536-959e-f250cd5fc883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.707178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9022a1-e78e-4536-959e-f250cd5fc883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.707583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9022a1-e78e-4536-959e-f250cd5fc883 admin admin] PATH_INFO: `/v3/projects/02636e4224fc4543bc39f718a57aa73c/users/55fd6ab0b4ae4661b0b86133c5fb689a/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.709773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9022a1-e78e-4536-959e-f250cd5fc883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02636e4224fc4543bc39f718a57aa73c, user_id=55fd6ab0b4ae4661b0b86133c5fb689a, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:00.714370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9022a1-e78e-4536-959e-f250cd5fc883 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:00.728740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 825/1621] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:00 2022] PUT /identity/v3/projects/02636e4224fc4543bc39f718a57aa73c/users/55fd6ab0b4ae4661b0b86133c5fb689a/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:00.738865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f908f4-b548-403f-9d51-3d314989546d None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.738865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f908f4-b548-403f-9d51-3d314989546d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.738865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f908f4-b548-403f-9d51-3d314989546d None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.738865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 826/1622] 10.222.0.64 () {54 vars in 961 bytes} [Mon Jun 27 09:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:36:00.738865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f908f4-b548-403f-9d51-3d314989546d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:00.738865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-59f908f4-b548-403f-9d51-3d314989546d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:00.762910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 797/1623] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:00 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:00.764467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020bf54-494c-4331-978b-9784c2950bc5 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.764759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020bf54-494c-4331-978b-9784c2950bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.765842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020bf54-494c-4331-978b-9784c2950bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.786492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-6020bf54-494c-4331-978b-9784c2950bc5 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:00.787758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6020bf54-494c-4331-978b-9784c2950bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:00.807782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f908f4-b548-403f-9d51-3d314989546d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:00.859982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ad48-169b-4d6d-86cb-e7f79082648b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:00.860651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ad48-169b-4d6d-86cb-e7f79082648b service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.860933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ad48-169b-4d6d-86cb-e7f79082648b service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.861385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ad48-169b-4d6d-86cb-e7f79082648b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.864355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 798/1624] 10.222.0.64 () {58 vars in 1058 bytes} [Mon Jun 27 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:00.865912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa25d45-d741-4c33-a063-5559028427ef None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:00.866206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa25d45-d741-4c33-a063-5559028427ef None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:00.866602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa25d45-d741-4c33-a063-5559028427ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:00.896624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-efa25d45-d741-4c33-a063-5559028427ef None None] MFA Rules not processed for user `55fd6ab0b4ae4661b0b86133c5fb689a`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:00.898027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-efa25d45-d741-4c33-a063-5559028427ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:00.949315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838ad48-169b-4d6d-86cb-e7f79082648b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:00.950783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838ad48-169b-4d6d-86cb-e7f79082648b service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:00.961680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 827/1625] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:00.969317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ad48-169b-4d6d-86cb-e7f79082648b service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:00.986602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ad48-169b-4d6d-86cb-e7f79082648b service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:00.995232 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 799/1626] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:00.997233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa25d45-d741-4c33-a063-5559028427ef None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:01.001989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b8ba7-8ac9-49da-9481-ec52702c7297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:01.002716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b8ba7-8ac9-49da-9481-ec52702c7297 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.003198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b8ba7-8ac9-49da-9481-ec52702c7297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.003485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b8ba7-8ac9-49da-9481-ec52702c7297 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.004373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b8ba7-8ac9-49da-9481-ec52702c7297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:01.006845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b8ba7-8ac9-49da-9481-ec52702c7297 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:01.023777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa25d45-d741-4c33-a063-5559028427ef None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:01.031089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f37b3b-f9f0-48b6-a5e3-3dcbf67d7ee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:01.031812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f37b3b-f9f0-48b6-a5e3-3dcbf67d7ee1 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.032096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f37b3b-f9f0-48b6-a5e3-3dcbf67d7ee1 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.032373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f37b3b-f9f0-48b6-a5e3-3dcbf67d7ee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.098179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 828/1627] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:00 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:01.102331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b8ba7-8ac9-49da-9481-ec52702c7297 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:01.109951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:01.118389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f37b3b-f9f0-48b6-a5e3-3dcbf67d7ee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:01.120433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f37b3b-f9f0-48b6-a5e3-3dcbf67d7ee1 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:01.131656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 800/1628] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:01.132600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f37b3b-f9f0-48b6-a5e3-3dcbf67d7ee1 service nova] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:01.143944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f37b3b-f9f0-48b6-a5e3-3dcbf67d7ee1 service nova] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:01.164696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a7b8-0eea-4e52-a08d-35d2f7b6b7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:01.165668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a7b8-0eea-4e52-a08d-35d2f7b6b7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.166365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a7b8-0eea-4e52-a08d-35d2f7b6b7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.167290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a7b8-0eea-4e52-a08d-35d2f7b6b7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.168948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668a7b8-0eea-4e52-a08d-35d2f7b6b7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:01.171947 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668a7b8-0eea-4e52-a08d-35d2f7b6b7a3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:01.201155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 801/1629] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:01.201155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a7b8-0eea-4e52-a08d-35d2f7b6b7a3 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:01.215766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a7b8-0eea-4e52-a08d-35d2f7b6b7a3 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:01.237898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8635580-9680-45af-ae9d-6cc8cfafd795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:01.240253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635580-9680-45af-ae9d-6cc8cfafd795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.241207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635580-9680-45af-ae9d-6cc8cfafd795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.241659 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635580-9680-45af-ae9d-6cc8cfafd795 admin admin] PATH_INFO: `/v3/projects/3683b8b5be5c4e7fb537bdb3cb519643/users/3c264cc596584b66b002686f731e10a4/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.244403 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b8ba7-8ac9-49da-9481-ec52702c7297 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:01.254263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2369556-5f52-4cec-89a1-4b916881ac31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:01.256811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2369556-5f52-4cec-89a1-4b916881ac31 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.256811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2369556-5f52-4cec-89a1-4b916881ac31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.256811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2369556-5f52-4cec-89a1-4b916881ac31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.267623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8635580-9680-45af-ae9d-6cc8cfafd795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3683b8b5be5c4e7fb537bdb3cb519643, user_id=3c264cc596584b66b002686f731e10a4, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:01.271165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8635580-9680-45af-ae9d-6cc8cfafd795 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:01.303395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:01.313447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2369556-5f52-4cec-89a1-4b916881ac31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:01.316134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2369556-5f52-4cec-89a1-4b916881ac31 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:01.347063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 829/1630] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:01.406336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 802/1631] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:01 2022] PUT /identity/v3/projects/3683b8b5be5c4e7fb537bdb3cb519643/users/3c264cc596584b66b002686f731e10a4/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:01.410152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2369556-5f52-4cec-89a1-4b916881ac31 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:01.433441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2369556-5f52-4cec-89a1-4b916881ac31 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:01.531610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17991acd-d80d-40df-a03d-bde2464ee56f None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.531932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17991acd-d80d-40df-a03d-bde2464ee56f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.532113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17991acd-d80d-40df-a03d-bde2464ee56f None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.534119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 803/1632] 10.222.0.64 () {54 vars in 912 bytes} [Mon Jun 27 09:36:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:36:01.555959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5633ee4-2cae-41ee-8e3f-e320ff0d41e6 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.556228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5633ee4-2cae-41ee-8e3f-e320ff0d41e6 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.556514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5633ee4-2cae-41ee-8e3f-e320ff0d41e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.573976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002f225-10f4-47a7-be8f-8d700544a5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:01.574674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002f225-10f4-47a7-be8f-8d700544a5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.580392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002f225-10f4-47a7-be8f-8d700544a5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.580392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002f225-10f4-47a7-be8f-8d700544a5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.580392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002f225-10f4-47a7-be8f-8d700544a5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:01.584899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002f225-10f4-47a7-be8f-8d700544a5e7 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:01.595848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e5633ee4-2cae-41ee-8e3f-e320ff0d41e6 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:01.597359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e5633ee4-2cae-41ee-8e3f-e320ff0d41e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:01.604111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 830/1633] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:01.613115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002f225-10f4-47a7-be8f-8d700544a5e7 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:01.624312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002f225-10f4-47a7-be8f-8d700544a5e7 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:01.639445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862b90d-8a07-4acc-8e8e-d2d8603e69e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:01.640418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862b90d-8a07-4acc-8e8e-d2d8603e69e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.640756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862b90d-8a07-4acc-8e8e-d2d8603e69e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.641037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862b90d-8a07-4acc-8e8e-d2d8603e69e1 admin admin] PATH_INFO: `/v3/projects/3683b8b5be5c4e7fb537bdb3cb519643/users/3c264cc596584b66b002686f731e10a4/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.643118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862b90d-8a07-4acc-8e8e-d2d8603e69e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3683b8b5be5c4e7fb537bdb3cb519643, user_id=3c264cc596584b66b002686f731e10a4, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:01.668448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862b90d-8a07-4acc-8e8e-d2d8603e69e1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:01.746794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 831/1634] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:01 2022] PUT /identity/v3/projects/3683b8b5be5c4e7fb537bdb3cb519643/users/3c264cc596584b66b002686f731e10a4/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:01.761942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd279-b9d2-4c04-b511-6eae1a23a28f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.762432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd279-b9d2-4c04-b511-6eae1a23a28f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.762872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd279-b9d2-4c04-b511-6eae1a23a28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.815431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 804/1635] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:36:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:01.816758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5633ee4-2cae-41ee-8e3f-e320ff0d41e6 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:01.833700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7d9bd279-b9d2-4c04-b511-6eae1a23a28f None None] MFA Rules not processed for user `3c264cc596584b66b002686f731e10a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:01.835204 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7d9bd279-b9d2-4c04-b511-6eae1a23a28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:01.921314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5633ee4-2cae-41ee-8e3f-e320ff0d41e6 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:01.979218 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486bec-fc48-4e87-8ee5-b5a84a073645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:01.980296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486bec-fc48-4e87-8ee5-b5a84a073645 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:01.980296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486bec-fc48-4e87-8ee5-b5a84a073645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:01.980618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486bec-fc48-4e87-8ee5-b5a84a073645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:01.986027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 832/1636] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:01 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:01.986697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bd279-b9d2-4c04-b511-6eae1a23a28f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:02.003336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bd279-b9d2-4c04-b511-6eae1a23a28f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:02.010345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3509c8b-a994-4f53-8782-08d3f03de344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:02.011235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3509c8b-a994-4f53-8782-08d3f03de344 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:02.011750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3509c8b-a994-4f53-8782-08d3f03de344 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:02.012191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3509c8b-a994-4f53-8782-08d3f03de344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:02.032656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:02.085824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd486bec-fc48-4e87-8ee5-b5a84a073645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:02.088145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd486bec-fc48-4e87-8ee5-b5a84a073645 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:02.099882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 805/1637] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:01 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:02.167313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:02.168552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3509c8b-a994-4f53-8782-08d3f03de344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:02.170287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3509c8b-a994-4f53-8782-08d3f03de344 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:02.186868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 833/1638] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:02.680123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486bec-fc48-4e87-8ee5-b5a84a073645 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:02.690569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486bec-fc48-4e87-8ee5-b5a84a073645 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:02.697232 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30963ae9-b9ea-4adc-a2ec-81cd40585da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:02.697997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30963ae9-b9ea-4adc-a2ec-81cd40585da9 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:02.698228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30963ae9-b9ea-4adc-a2ec-81cd40585da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:02.698550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30963ae9-b9ea-4adc-a2ec-81cd40585da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:02.714162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30963ae9-b9ea-4adc-a2ec-81cd40585da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:02.714162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30963ae9-b9ea-4adc-a2ec-81cd40585da9 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:02.724868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 806/1639] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:02 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:02.951980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797899c5-13fc-49ab-ad3c-1c4e4764a22a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:02.951980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797899c5-13fc-49ab-ad3c-1c4e4764a22a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:02.951980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797899c5-13fc-49ab-ad3c-1c4e4764a22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:02.972006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-797899c5-13fc-49ab-ad3c-1c4e4764a22a None None] MFA Rules not processed for user `76bf077a5259406f81aff8af3b2b3a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:02.973340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-797899c5-13fc-49ab-ad3c-1c4e4764a22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:03.124502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 834/1640] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:03.128727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30963ae9-b9ea-4adc-a2ec-81cd40585da9 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:03.262558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30963ae9-b9ea-4adc-a2ec-81cd40585da9 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:03.274628 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5755a93-b083-4c66-b4d5-9f9cc261f488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:03.275887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5755a93-b083-4c66-b4d5-9f9cc261f488 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:03.275887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5755a93-b083-4c66-b4d5-9f9cc261f488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:03.276317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5755a93-b083-4c66-b4d5-9f9cc261f488 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:03.277350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5755a93-b083-4c66-b4d5-9f9cc261f488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:03.279741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5755a93-b083-4c66-b4d5-9f9cc261f488 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:03.686773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 807/1641] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:03 2022] POST /identity/v3/projects => generated 405 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:03.691822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797899c5-13fc-49ab-ad3c-1c4e4764a22a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:03.691822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797899c5-13fc-49ab-ad3c-1c4e4764a22a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:03.691822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea96a0d-1d9a-466c-8887-37dc520c0acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:03.691822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea96a0d-1d9a-466c-8887-37dc520c0acd admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:03.694464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea96a0d-1d9a-466c-8887-37dc520c0acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:03.694464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea96a0d-1d9a-466c-8887-37dc520c0acd admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:03.694464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea96a0d-1d9a-466c-8887-37dc520c0acd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:03.694464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea96a0d-1d9a-466c-8887-37dc520c0acd admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:03.743159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0685d1-08c5-488b-91e2-9e33136ccc15 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:03.743450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0685d1-08c5-488b-91e2-9e33136ccc15 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:03.743792 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0685d1-08c5-488b-91e2-9e33136ccc15 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:03.745443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 808/1642] 10.222.0.64 () {54 vars in 961 bytes} [Mon Jun 27 09:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:36:03.752606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1269e-a24c-4e4a-9d5e-fa24a45ede00 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:03.753431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1269e-a24c-4e4a-9d5e-fa24a45ede00 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:03.753852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1269e-a24c-4e4a-9d5e-fa24a45ede00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:03.779127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-2fb1269e-a24c-4e4a-9d5e-fa24a45ede00 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:03.780642 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2fb1269e-a24c-4e4a-9d5e-fa24a45ede00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:03.876515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 809/1643] 10.222.0.64 () {58 vars in 1058 bytes} [Mon Jun 27 09:36:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:03.882016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c6e29-5a60-4732-b5be-d1c79e2bb13e None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:03.883256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c6e29-5a60-4732-b5be-d1c79e2bb13e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:03.883902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c6e29-5a60-4732-b5be-d1c79e2bb13e None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:03.885790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 810/1644] 10.222.0.64 () {54 vars in 912 bytes} [Mon Jun 27 09:36:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 09:36:03.894648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d61d-edc8-4653-83ef-a23f0fd118b4 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:03.896769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d61d-edc8-4653-83ef-a23f0fd118b4 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:03.896769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d61d-edc8-4653-83ef-a23f0fd118b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:03.918551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-02c7d61d-edc8-4653-83ef-a23f0fd118b4 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:03.922533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-02c7d61d-edc8-4653-83ef-a23f0fd118b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:03.932588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 835/1645] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:03 2022] POST /identity/v3/users => generated 309 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:03.935909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea96a0d-1d9a-466c-8887-37dc520c0acd admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:03.951345 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 811/1646] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:36:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:03.978777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea96a0d-1d9a-466c-8887-37dc520c0acd admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:03.993987 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e54a5-1278-45a7-ac78-8747a36597be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:03.995080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e54a5-1278-45a7-ac78-8747a36597be admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:03.995551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e54a5-1278-45a7-ac78-8747a36597be admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:03.995965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e54a5-1278-45a7-ac78-8747a36597be admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:04.000114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e54a5-1278-45a7-ac78-8747a36597be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:04.009777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e54a5-1278-45a7-ac78-8747a36597be admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:04.040839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 836/1647] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:04.043070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d61d-edc8-4653-83ef-a23f0fd118b4 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:04.078964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d61d-edc8-4653-83ef-a23f0fd118b4 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:04.100296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c780094-7e7e-4425-a508-b00665a33368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:04.102978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c780094-7e7e-4425-a508-b00665a33368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:04.105747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c780094-7e7e-4425-a508-b00665a33368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:04.108061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c780094-7e7e-4425-a508-b00665a33368 admin admin] PATH_INFO: `/v3/projects/e0af33cde1b7425181dfacd9ef8fa830/users/fcddc3b1ff5f432f85dd6b1e343df0cf/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:04.162187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c780094-7e7e-4425-a508-b00665a33368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0af33cde1b7425181dfacd9ef8fa830, user_id=fcddc3b1ff5f432f85dd6b1e343df0cf, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:04.166961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c780094-7e7e-4425-a508-b00665a33368 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:04.214447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e54a5-1278-45a7-ac78-8747a36597be admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:04.233937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:04.274304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:04.286957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 812/1648] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:04 2022] PUT /identity/v3/projects/e0af33cde1b7425181dfacd9ef8fa830/users/fcddc3b1ff5f432f85dd6b1e343df0cf/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:04.289092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c780094-7e7e-4425-a508-b00665a33368 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:04.381045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e54a5-1278-45a7-ac78-8747a36597be admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:04.421679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c780094-7e7e-4425-a508-b00665a33368 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:04.436225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9e6db-3432-4021-ace7-465e87b78ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:04.437059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9e6db-3432-4021-ace7-465e87b78ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:04.437544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9e6db-3432-4021-ace7-465e87b78ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:04.437884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9e6db-3432-4021-ace7-465e87b78ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:04.438793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9e6db-3432-4021-ace7-465e87b78ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:04.442504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9e6db-3432-4021-ace7-465e87b78ed1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:04.469042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f098f5-2a60-43e0-b340-1269bd3c5a8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:04.469931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 813/1649] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:04.470116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f098f5-2a60-43e0-b340-1269bd3c5a8e service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:04.470379 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f098f5-2a60-43e0-b340-1269bd3c5a8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:04.470665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f098f5-2a60-43e0-b340-1269bd3c5a8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:04.471285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9e6db-3432-4021-ace7-465e87b78ed1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:04.494665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9e6db-3432-4021-ace7-465e87b78ed1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:04.502338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7d2c3-f577-4633-a57f-1b40fd58d58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:04.503291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7d2c3-f577-4633-a57f-1b40fd58d58a admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:04.503624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7d2c3-f577-4633-a57f-1b40fd58d58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:04.503951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7d2c3-f577-4633-a57f-1b40fd58d58a admin admin] PATH_INFO: `/v3/projects/e0af33cde1b7425181dfacd9ef8fa830/users/fcddc3b1ff5f432f85dd6b1e343df0cf/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:04.506192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7d2c3-f577-4633-a57f-1b40fd58d58a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0af33cde1b7425181dfacd9ef8fa830, user_id=fcddc3b1ff5f432f85dd6b1e343df0cf, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:04.509267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7d2c3-f577-4633-a57f-1b40fd58d58a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:04.526049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 814/1650] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:04 2022] PUT /identity/v3/projects/e0af33cde1b7425181dfacd9ef8fa830/users/fcddc3b1ff5f432f85dd6b1e343df0cf/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:04.567284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f098f5-2a60-43e0-b340-1269bd3c5a8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:04.571687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f098f5-2a60-43e0-b340-1269bd3c5a8e service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:04.599439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 837/1651] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:08.989745 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a83aa-c8a9-4148-8d2e-b166ad89f1a3 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:08.990741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a83aa-c8a9-4148-8d2e-b166ad89f1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:08.991824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a83aa-c8a9-4148-8d2e-b166ad89f1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:09.043423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-691a83aa-c8a9-4148-8d2e-b166ad89f1a3 None None] MFA Rules not processed for user `fcddc3b1ff5f432f85dd6b1e343df0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:09.045779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-691a83aa-c8a9-4148-8d2e-b166ad89f1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:09.305604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 815/1652] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:08 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:09.649740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f098f5-2a60-43e0-b340-1269bd3c5a8e service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:09.674626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f098f5-2a60-43e0-b340-1269bd3c5a8e service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:09.681632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77e05b-d245-4960-81f8-9f94c77de137 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:09.681932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77e05b-d245-4960-81f8-9f94c77de137 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:09.682220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77e05b-d245-4960-81f8-9f94c77de137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:09.758491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-6a77e05b-d245-4960-81f8-9f94c77de137 None None] MFA Rules not processed for user `ecf085030e8e4a65a7140fea2b64a7ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:09.763643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6a77e05b-d245-4960-81f8-9f94c77de137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:09.867162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:09.868233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:09.877281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:09.877281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] PATH_INFO: `/v3/users/d11f89ecadc5445ca2b92bf4592f1b04` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:09.881348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d11f89ecadc5445ca2b92bf4592f1b04)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:09.895292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:09.911489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:09.948342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:09.991181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 816/1653] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:09.997292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77e05b-d245-4960-81f8-9f94c77de137 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:10.146119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd11f89ecadc5445ca2b92bf4592f1b04'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:10.192497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77e05b-d245-4960-81f8-9f94c77de137 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:10.201121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874eab4c-b66c-4598-b242-ddfb9789c747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:10.201811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eab4c-b66c-4598-b242-ddfb9789c747 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:10.202098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eab4c-b66c-4598-b242-ddfb9789c747 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:10.202411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eab4c-b66c-4598-b242-ddfb9789c747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:10.223793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874eab4c-b66c-4598-b242-ddfb9789c747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:10.227727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874eab4c-b66c-4598-b242-ddfb9789c747 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:10.243110 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 817/1654] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:10.257145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd11f89ecadc5445ca2b92bf4592f1b04'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:10.258823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd11f89ecadc5445ca2b92bf4592f1b04'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:10.273430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd11f89ecadc5445ca2b92bf4592f1b04'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:10.277615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd11f89ecadc5445ca2b92bf4592f1b04'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:10.317552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 838/1655] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:09 2022] DELETE /identity/v3/users/d11f89ecadc5445ca2b92bf4592f1b04 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:10.321121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1926950-bdcd-40d4-bd05-5beea1bb2a24 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:10.321779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1926950-bdcd-40d4-bd05-5beea1bb2a24 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:10.322173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1926950-bdcd-40d4-bd05-5beea1bb2a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:10.350632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e1926950-bdcd-40d4-bd05-5beea1bb2a24 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:10.350632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e1926950-bdcd-40d4-bd05-5beea1bb2a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:10.503364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 818/1656] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:10.518576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:10.523789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:10.668305 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b45c7e-8059-4302-93a6-e636cac0fabe admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:10.678242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1be21-1fba-43d9-ab96-c601fb4804ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:10.678932 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1be21-1fba-43d9-ab96-c601fb4804ae service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:10.679571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1be21-1fba-43d9-ab96-c601fb4804ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:10.679998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1be21-1fba-43d9-ab96-c601fb4804ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:10.711091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1be21-1fba-43d9-ab96-c601fb4804ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:10.712995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1be21-1fba-43d9-ab96-c601fb4804ae service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:10.733707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 839/1657] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:12.066786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626df98-0cc5-4630-a895-a4db2650eaa7 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:12.067256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626df98-0cc5-4630-a895-a4db2650eaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:12.067474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626df98-0cc5-4630-a895-a4db2650eaa7 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:12.069137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 819/1658] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:36:12.100433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ea83c-c9b7-44bf-acf6-0abfb0db622e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:12.100433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ea83c-c9b7-44bf-acf6-0abfb0db622e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:12.100433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ea83c-c9b7-44bf-acf6-0abfb0db622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:12.149065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e59ea83c-c9b7-44bf-acf6-0abfb0db622e None None] MFA Rules not processed for user `19d4d5fa82dc42e498fb22349921885e`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:12.152729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e59ea83c-c9b7-44bf-acf6-0abfb0db622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:12.341668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 840/1659] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:36:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:12.477977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ced4b-e562-43b7-bbcc-887d67335fc4 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:12.478462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ced4b-e562-43b7-bbcc-887d67335fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:12.478907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ced4b-e562-43b7-bbcc-887d67335fc4 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:12.481082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 820/1660] 10.222.0.64 () {54 vars in 960 bytes} [Mon Jun 27 09:36:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:36:12.488137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb855b-c5c2-4e52-a74b-021099725d5c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:12.488433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb855b-c5c2-4e52-a74b-021099725d5c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:12.488716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb855b-c5c2-4e52-a74b-021099725d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:12.526840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-fffb855b-c5c2-4e52-a74b-021099725d5c None None] MFA Rules not processed for user `19d4d5fa82dc42e498fb22349921885e`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:12.528387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fffb855b-c5c2-4e52-a74b-021099725d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:12.553572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ced4b-e562-43b7-bbcc-887d67335fc4 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:12.558559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 841/1661] 10.222.0.64 () {58 vars in 1057 bytes} [Mon Jun 27 09:36:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:12.569626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ced4b-e562-43b7-bbcc-887d67335fc4 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:12.576454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da90c0-992d-496b-adc7-96a693a32fa0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:12.577509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da90c0-992d-496b-adc7-96a693a32fa0 service placement] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:12.578003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da90c0-992d-496b-adc7-96a693a32fa0 service placement] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:12.578529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da90c0-992d-496b-adc7-96a693a32fa0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:12.591009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da90c0-992d-496b-adc7-96a693a32fa0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:12.593180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da90c0-992d-496b-adc7-96a693a32fa0 service placement] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:12.622012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 821/1662] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:12.711117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb855b-c5c2-4e52-a74b-021099725d5c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:12.711117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fffb855b-c5c2-4e52-a74b-021099725d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:12.750005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb855b-c5c2-4e52-a74b-021099725d5c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:12.759147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:12.762248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:12.762865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:12.763373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] PATH_INFO: `/v3/projects/72b88c3f520e421bbfabac2620d19f0b` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:12.766622 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72b88c3f520e421bbfabac2620d19f0b)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:12.768122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: WARNING py.warnings [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:36:12.768122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: warnings.warn(msg) Jun 27 09:36:12.768122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: Jun 27 09:36:12.771373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:12.791457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:12.979848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72b88c3f520e421bbfabac2620d19f0b', 'request_id': 'req-32856679-e25e-4d49-81b4-fa4611b7a987'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:13.113993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72b88c3f520e421bbfabac2620d19f0b', 'request_id': 'req-32856679-e25e-4d49-81b4-fa4611b7a987'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:13.123490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72b88c3f520e421bbfabac2620d19f0b', 'request_id': 'req-32856679-e25e-4d49-81b4-fa4611b7a987'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:13.197770 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] The token cache is being invalidate because project 72b88c3f520e421bbfabac2620d19f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:13.203834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:13.251913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:13.260177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 842/1663] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:12 2022] DELETE /identity/v3/projects/72b88c3f520e421bbfabac2620d19f0b => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:13.267661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da90c0-992d-496b-adc7-96a693a32fa0 service placement] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:13.298782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da90c0-992d-496b-adc7-96a693a32fa0 service placement] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:13.526631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:13.529545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:13.530168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:13.530841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] PATH_INFO: `/v3/users/3c264cc596584b66b002686f731e10a4` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:13.532756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c264cc596584b66b002686f731e10a4)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:13.533922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: WARNING py.warnings [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 09:36:13.533922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: warnings.warn(msg) Jun 27 09:36:13.533922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: Jun 27 09:36:13.535944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:13.551918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:13.890840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c264cc596584b66b002686f731e10a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:13.892826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c264cc596584b66b002686f731e10a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:13.932123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c264cc596584b66b002686f731e10a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:13.933851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c264cc596584b66b002686f731e10a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:13.998796 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c264cc596584b66b002686f731e10a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:14.068908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:14.071265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:14.113204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 822/1664] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:13 2022] DELETE /identity/v3/users/3c264cc596584b66b002686f731e10a4 => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:14.127093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32856679-e25e-4d49-81b4-fa4611b7a987 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:14.363940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:14.365164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:14.367177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:14.370873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] PATH_INFO: `/v3/users/76bf077a5259406f81aff8af3b2b3a02` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:14.384319 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76bf077a5259406f81aff8af3b2b3a02)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:14.388829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:14.750168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76bf077a5259406f81aff8af3b2b3a02'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:14.769748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:14.770686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:14.808455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace803d-e619-4bfa-a822-94a327e8ec27 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:14.828516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:14.829656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:14.830031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:14.830850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] PATH_INFO: `/v3/users/0e7a041f66e245189135fe8bedc93448` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:14.832333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e7a041f66e245189135fe8bedc93448)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:14.834518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:14.885905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76bf077a5259406f81aff8af3b2b3a02'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:14.886681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76bf077a5259406f81aff8af3b2b3a02'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:14.905433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76bf077a5259406f81aff8af3b2b3a02'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:14.906491 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9210c152-d208-4948-b09d-bc36c51972b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76bf077a5259406f81aff8af3b2b3a02'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:14.952232 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 843/1665] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:14 2022] DELETE /identity/v3/users/76bf077a5259406f81aff8af3b2b3a02 => generated 0 bytes in 883 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:15.089211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e7a041f66e245189135fe8bedc93448'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:15.089926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e7a041f66e245189135fe8bedc93448'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:15.109566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e7a041f66e245189135fe8bedc93448'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:15.110589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e7a041f66e245189135fe8bedc93448'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:15.145774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-49f26688-7d13-4b4a-a62c-8a370ba7ec8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e7a041f66e245189135fe8bedc93448'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:15.245247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f72e5-3042-4793-8009-71a2d657b947 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:15.246167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f72e5-3042-4793-8009-71a2d657b947 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:15.246582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f72e5-3042-4793-8009-71a2d657b947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:15.261715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 823/1666] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:14 2022] DELETE /identity/v3/users/0e7a041f66e245189135fe8bedc93448 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:15.263431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebf136-9ff2-4a58-819f-222b65c9931d None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:15.265489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebf136-9ff2-4a58-819f-222b65c9931d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:15.266015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebf136-9ff2-4a58-819f-222b65c9931d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:15.310159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-703f72e5-3042-4793-8009-71a2d657b947 None None] MFA Rules not processed for user `41ce5cc7ff504d64aac73e48097a4b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:15.315149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-703f72e5-3042-4793-8009-71a2d657b947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:15.328045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:15.330948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a9ebf136-9ff2-4a58-819f-222b65c9931d None None] MFA Rules not processed for user `9b5d5a3f43c544368118125f2e194565`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:15.332803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a9ebf136-9ff2-4a58-819f-222b65c9931d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:15.483052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:15.513451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 844/1667] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:15.515185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f743a10-3a13-4d74-82c6-a61e838ade38 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:15.531050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f743a10-3a13-4d74-82c6-a61e838ade38 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:15.531050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f743a10-3a13-4d74-82c6-a61e838ade38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:15.531417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 824/1668] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:15.548821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683ac22-ed51-4162-976b-5268b4bfff1f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:15.549125 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683ac22-ed51-4162-976b-5268b4bfff1f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:15.549399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683ac22-ed51-4162-976b-5268b4bfff1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:15.576061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4f743a10-3a13-4d74-82c6-a61e838ade38 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:15.580662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4f743a10-3a13-4d74-82c6-a61e838ade38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:15.614132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7683ac22-ed51-4162-976b-5268b4bfff1f None None] MFA Rules not processed for user `41ce5cc7ff504d64aac73e48097a4b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:15.616214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7683ac22-ed51-4162-976b-5268b4bfff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:15.626005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 825/1669] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:15.639209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4471c3-a3aa-430e-ac40-9d421b787f9c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:15.640335 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4471c3-a3aa-430e-ac40-9d421b787f9c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:15.640823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4471c3-a3aa-430e-ac40-9d421b787f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:15.741512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 845/1670] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:15.743058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f743a10-3a13-4d74-82c6-a61e838ade38 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:15.750076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4f743a10-3a13-4d74-82c6-a61e838ade38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:15.785426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f743a10-3a13-4d74-82c6-a61e838ade38 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:15.802246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:15.803156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:15.803497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:15.803497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] PATH_INFO: `/v3/projects/3683b8b5be5c4e7fb537bdb3cb519643` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:15.809342 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3683b8b5be5c4e7fb537bdb3cb519643)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:15.813429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:15.948681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7b4471c3-a3aa-430e-ac40-9d421b787f9c None None] MFA Rules not processed for user `9b5d5a3f43c544368118125f2e194565`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:15.950899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7b4471c3-a3aa-430e-ac40-9d421b787f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:15.959956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 826/1671] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:15.961635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4471c3-a3aa-430e-ac40-9d421b787f9c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:15.963199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7b4471c3-a3aa-430e-ac40-9d421b787f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:16.063011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3683b8b5be5c4e7fb537bdb3cb519643', 'request_id': 'req-a073166e-d554-4586-9cbb-a1aa5ac5442f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.100678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4471c3-a3aa-430e-ac40-9d421b787f9c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:16.153920 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3683b8b5be5c4e7fb537bdb3cb519643', 'request_id': 'req-a073166e-d554-4586-9cbb-a1aa5ac5442f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.164218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3683b8b5be5c4e7fb537bdb3cb519643', 'request_id': 'req-a073166e-d554-4586-9cbb-a1aa5ac5442f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.192851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] The token cache is being invalidate because project 3683b8b5be5c4e7fb537bdb3cb519643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:16.193146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.194289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:16.195415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:16.196224 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:16.196514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.196636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:16.199659 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:16.200604 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 846/1672] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:15 2022] DELETE /identity/v3/projects/3683b8b5be5c4e7fb537bdb3cb519643 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:16.201224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:16.215109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:16.294204 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:16.346784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073166e-d554-4586-9cbb-a1aa5ac5442f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:16.352678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:16.353735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:16.359281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:16.359281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:16.359281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:16.361466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:16.366775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:16.401121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 827/1673] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:16.406438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:16.411578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:16.495228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:16.496898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:16.529642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 847/1674] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:16.542068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:16.545409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:16.583273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2c13b-742a-474c-96fe-f12541f61d3f service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:16.586189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d6cca-a04e-42fe-a5d3-184653409be0 service nova] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:16.602870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522eb35-3f79-49f0-ae3f-98275f5cd65d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:16.603592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522eb35-3f79-49f0-ae3f-98275f5cd65d service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:16.606329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522eb35-3f79-49f0-ae3f-98275f5cd65d service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:16.606777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522eb35-3f79-49f0-ae3f-98275f5cd65d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:16.606935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:16.607863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:16.608148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:16.608424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] PATH_INFO: `/v3/projects/e93273a70b35491d8d0acf14ddb5afe4` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:16.611167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93273a70b35491d8d0acf14ddb5afe4)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:16.612103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6522eb35-3f79-49f0-ae3f-98275f5cd65d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:16.613160 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:16.742030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522eb35-3f79-49f0-ae3f-98275f5cd65d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:16.743992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522eb35-3f79-49f0-ae3f-98275f5cd65d service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:16.767406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 848/1675] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:16.769467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442fa5f-6e91-4838-b152-a0001ac94312 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:16.769467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442fa5f-6e91-4838-b152-a0001ac94312 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:16.769693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442fa5f-6e91-4838-b152-a0001ac94312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:16.804825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93273a70b35491d8d0acf14ddb5afe4', 'request_id': 'req-276b2c66-1b99-479f-9924-15a356c103f4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.811576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93273a70b35491d8d0acf14ddb5afe4', 'request_id': 'req-276b2c66-1b99-479f-9924-15a356c103f4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.819285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93273a70b35491d8d0acf14ddb5afe4', 'request_id': 'req-276b2c66-1b99-479f-9924-15a356c103f4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.838747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-6442fa5f-6e91-4838-b152-a0001ac94312 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:16.840112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6442fa5f-6e91-4838-b152-a0001ac94312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:16.903389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] The token cache is being invalidate because project e93273a70b35491d8d0acf14ddb5afe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:16.903962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.904652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-276b2c66-1b99-479f-9924-15a356c103f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:16.913159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 828/1676] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:16 2022] DELETE /identity/v3/projects/e93273a70b35491d8d0acf14ddb5afe4 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:16.915780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4051f74-230d-4b2d-a280-9189287024c5 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:16.921840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4051f74-230d-4b2d-a280-9189287024c5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:16.921840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4051f74-230d-4b2d-a280-9189287024c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:16.973091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f4051f74-230d-4b2d-a280-9189287024c5 None None] MFA Rules not processed for user `ecf085030e8e4a65a7140fea2b64a7ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:16.975288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f4051f74-230d-4b2d-a280-9189287024c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:17.014663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 849/1677] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:17.018255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442fa5f-6e91-4838-b152-a0001ac94312 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:17.021947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6442fa5f-6e91-4838-b152-a0001ac94312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:17.051722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442fa5f-6e91-4838-b152-a0001ac94312 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:17.067274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:17.073065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.074498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.074817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] PATH_INFO: `/v3/users/fcddc3b1ff5f432f85dd6b1e343df0cf` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.078927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcddc3b1ff5f432f85dd6b1e343df0cf)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:17.082727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:17.131464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 829/1678] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:17.134694 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5f61-0998-4f1a-9a48-556cac24f1ac None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.137060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5f61-0998-4f1a-9a48-556cac24f1ac None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.137392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5f61-0998-4f1a-9a48-556cac24f1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.173121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-841c5f61-0998-4f1a-9a48-556cac24f1ac None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:17.174385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-841c5f61-0998-4f1a-9a48-556cac24f1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:17.184670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 830/1679] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:17.188814 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5f61-0998-4f1a-9a48-556cac24f1ac None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:17.199547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5f61-0998-4f1a-9a48-556cac24f1ac None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:17.211709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabebf8-2b75-4a9e-967a-03b551ba7952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:17.212544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabebf8-2b75-4a9e-967a-03b551ba7952 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.212831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabebf8-2b75-4a9e-967a-03b551ba7952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.213109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabebf8-2b75-4a9e-967a-03b551ba7952 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.213868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabebf8-2b75-4a9e-967a-03b551ba7952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:17.214173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabebf8-2b75-4a9e-967a-03b551ba7952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:17.215960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabebf8-2b75-4a9e-967a-03b551ba7952 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:17.247201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 831/1680] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:36:17.248251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabebf8-2b75-4a9e-967a-03b551ba7952 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:17.261805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabebf8-2b75-4a9e-967a-03b551ba7952 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:17.267265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a79e9-9825-4f51-b770-ac949d37a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:17.272126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a79e9-9825-4f51-b770-ac949d37a9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.272126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a79e9-9825-4f51-b770-ac949d37a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.272126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a79e9-9825-4f51-b770-ac949d37a9f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.272126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a79e9-9825-4f51-b770-ac949d37a9f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:17.272126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a79e9-9825-4f51-b770-ac949d37a9f3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:17.416925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 832/1681] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:17 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:17.417757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27cb8f-2589-48b0-b86b-3cfe32a9910f None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.418353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27cb8f-2589-48b0-b86b-3cfe32a9910f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.418353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27cb8f-2589-48b0-b86b-3cfe32a9910f None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.425907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 833/1682] 10.222.0.64 () {54 vars in 912 bytes} [Mon Jun 27 09:36:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:36:17.428471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27cb8f-2589-48b0-b86b-3cfe32a9910f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:17.447523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcddc3b1ff5f432f85dd6b1e343df0cf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:17.598040 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcddc3b1ff5f432f85dd6b1e343df0cf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:17.598040 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcddc3b1ff5f432f85dd6b1e343df0cf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:17.606965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcddc3b1ff5f432f85dd6b1e343df0cf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:17.608050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8fce1fef-0a2f-4886-9885-9ad07dfab191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcddc3b1ff5f432f85dd6b1e343df0cf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:17.614534 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27cb8f-2589-48b0-b86b-3cfe32a9910f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:17.620422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:17.630240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112e9b-cc04-48c8-9eb5-99abdadc8211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:17.631744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112e9b-cc04-48c8-9eb5-99abdadc8211 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.632079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112e9b-cc04-48c8-9eb5-99abdadc8211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.632364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112e9b-cc04-48c8-9eb5-99abdadc8211 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.633080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112e9b-cc04-48c8-9eb5-99abdadc8211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:17.636540 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112e9b-cc04-48c8-9eb5-99abdadc8211 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:17.656809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 850/1683] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:17 2022] DELETE /identity/v3/users/fcddc3b1ff5f432f85dd6b1e343df0cf => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:17.662927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1fc1a-8ca7-4dc1-9ee0-bc1b846e4b10 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.664067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1fc1a-8ca7-4dc1-9ee0-bc1b846e4b10 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.665033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1fc1a-8ca7-4dc1-9ee0-bc1b846e4b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.751244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-fca1fc1a-8ca7-4dc1-9ee0-bc1b846e4b10 None None] MFA Rules not processed for user `2b87f03cae6c4ca98a82dd5d981ed224`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:17.753656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fca1fc1a-8ca7-4dc1-9ee0-bc1b846e4b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:17.822630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 834/1684] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:17 2022] POST /identity/v3/users => generated 290 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:17.823929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112e9b-cc04-48c8-9eb5-99abdadc8211 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:17.833296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c9112e9b-cc04-48c8-9eb5-99abdadc8211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:17.848169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112e9b-cc04-48c8-9eb5-99abdadc8211 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:17.867275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 851/1685] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:17.869273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb222df-a5c5-4bcf-b431-36a5cf4feb5d None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.869697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb222df-a5c5-4bcf-b431-36a5cf4feb5d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.870651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb222df-a5c5-4bcf-b431-36a5cf4feb5d None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.879674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 852/1686] 10.222.0.64 () {54 vars in 912 bytes} [Mon Jun 27 09:36:17 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:36:17.882329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01475ecc-ec25-4276-bba5-705887c823fd None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.886966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01475ecc-ec25-4276-bba5-705887c823fd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.886966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01475ecc-ec25-4276-bba5-705887c823fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.927622 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-01475ecc-ec25-4276-bba5-705887c823fd None None] MFA Rules not processed for user `2b87f03cae6c4ca98a82dd5d981ed224`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:17.934165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-01475ecc-ec25-4276-bba5-705887c823fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:17.937850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0387-7813-4a45-8b69-21e3ef26d5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:17.938730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0387-7813-4a45-8b69-21e3ef26d5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.939163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0387-7813-4a45-8b69-21e3ef26d5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.939654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0387-7813-4a45-8b69-21e3ef26d5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.940538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb0387-7813-4a45-8b69-21e3ef26d5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:17.945478 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb0387-7813-4a45-8b69-21e3ef26d5d0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:17.960686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 853/1687] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:17.961715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01475ecc-ec25-4276-bba5-705887c823fd None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:17.963213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 835/1688] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:17.964316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0387-7813-4a45-8b69-21e3ef26d5d0 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:17.968122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ddcb0387-7813-4a45-8b69-21e3ef26d5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:17.968967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-01475ecc-ec25-4276-bba5-705887c823fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:17.983276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01475ecc-ec25-4276-bba5-705887c823fd None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:17.984128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0387-7813-4a45-8b69-21e3ef26d5d0 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:17.988489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af2e6c-f4dc-43c6-8213-312d2f93a36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:17.989469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af2e6c-f4dc-43c6-8213-312d2f93a36e service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.993365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af2e6c-f4dc-43c6-8213-312d2f93a36e service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.993365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af2e6c-f4dc-43c6-8213-312d2f93a36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.993543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:17.993543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:17.993543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:17.993543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] PATH_INFO: `/v3/users/6ae4a199cf0f41a38eb6d4c283beebc5` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:17.994317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae4a199cf0f41a38eb6d4c283beebc5)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:17.996809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:18.001383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af2e6c-f4dc-43c6-8213-312d2f93a36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:18.004494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af2e6c-f4dc-43c6-8213-312d2f93a36e service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:18.018718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 854/1689] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:18.027633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af2e6c-f4dc-43c6-8213-312d2f93a36e service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:18.061371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af2e6c-f4dc-43c6-8213-312d2f93a36e service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:18.071097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e540342-05c9-4e49-b27d-e4e5b51c52e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:18.071957 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e540342-05c9-4e49-b27d-e4e5b51c52e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:18.074940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e540342-05c9-4e49-b27d-e4e5b51c52e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:18.075259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e540342-05c9-4e49-b27d-e4e5b51c52e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:18.076614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e540342-05c9-4e49-b27d-e4e5b51c52e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:18.083327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e540342-05c9-4e49-b27d-e4e5b51c52e8 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:18.108223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 855/1690] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:18.112862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e540342-05c9-4e49-b27d-e4e5b51c52e8 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:18.115031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5e540342-05c9-4e49-b27d-e4e5b51c52e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:18.133347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e540342-05c9-4e49-b27d-e4e5b51c52e8 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:18.137777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f791aa1-e98b-42cb-b766-11836b989093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:18.138489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791aa1-e98b-42cb-b766-11836b989093 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:18.138774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791aa1-e98b-42cb-b766-11836b989093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:18.139050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791aa1-e98b-42cb-b766-11836b989093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:18.147587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f791aa1-e98b-42cb-b766-11836b989093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:18.149840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f791aa1-e98b-42cb-b766-11836b989093 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:18.159786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 856/1691] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:18.160745 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f791aa1-e98b-42cb-b766-11836b989093 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:18.167711 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6f791aa1-e98b-42cb-b766-11836b989093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:18.201487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f791aa1-e98b-42cb-b766-11836b989093 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:18.206745 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:18.208050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:18.208332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:18.208707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] PATH_INFO: `/v3/projects/eb3c0fa49ea5424399304b9f9b18e866` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:18.210933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3c0fa49ea5424399304b9f9b18e866)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:18.213238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:18.229316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae4a199cf0f41a38eb6d4c283beebc5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.234631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae4a199cf0f41a38eb6d4c283beebc5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.249623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae4a199cf0f41a38eb6d4c283beebc5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.250761 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae4a199cf0f41a38eb6d4c283beebc5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.286459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae4a199cf0f41a38eb6d4c283beebc5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.385053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3c0fa49ea5424399304b9f9b18e866', 'request_id': 'req-64de4495-f1bd-4561-b635-4c7251abf892'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.395488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 836/1692] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:17 2022] DELETE /identity/v3/users/6ae4a199cf0f41a38eb6d4c283beebc5 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:18.395852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:18.401836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:18.492194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3c0fa49ea5424399304b9f9b18e866', 'request_id': 'req-64de4495-f1bd-4561-b635-4c7251abf892'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.499388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:18.506978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3c0fa49ea5424399304b9f9b18e866', 'request_id': 'req-64de4495-f1bd-4561-b635-4c7251abf892'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.541597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] The token cache is being invalidate because project eb3c0fa49ea5424399304b9f9b18e866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:18.541936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.542612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:18.544865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 857/1693] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:18 2022] DELETE /identity/v3/projects/eb3c0fa49ea5424399304b9f9b18e866 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:18.545871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:18.547365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:18.622806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c229b90-b5ed-4051-8fbd-cd2880f8d8ec admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:18.639442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27169f62-5c5c-4921-b58a-33d6c33c3726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:18.640429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27169f62-5c5c-4921-b58a-33d6c33c3726 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:18.640734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27169f62-5c5c-4921-b58a-33d6c33c3726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:18.641002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27169f62-5c5c-4921-b58a-33d6c33c3726 admin admin] PATH_INFO: `/v3/projects/aa7a1e861f514e8cbe29db155db43981/users/f8ee605bf3eb44e99d49af0f3a0f578a/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:18.666381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27169f62-5c5c-4921-b58a-33d6c33c3726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7a1e861f514e8cbe29db155db43981, user_id=f8ee605bf3eb44e99d49af0f3a0f578a, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:18.669671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27169f62-5c5c-4921-b58a-33d6c33c3726 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:18.734893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de4495-f1bd-4561-b635-4c7251abf892 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:18.743144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:18.745797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:18.746932 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:18.747876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:18.748451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] PATH_INFO: `/v3/projects/e0af33cde1b7425181dfacd9ef8fa830` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:18.759991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0af33cde1b7425181dfacd9ef8fa830)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:18.759991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:18.760195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 837/1694] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:18 2022] PUT /identity/v3/projects/aa7a1e861f514e8cbe29db155db43981/users/f8ee605bf3eb44e99d49af0f3a0f578a/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:18.760195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c9f60-9e86-4a40-b06e-831e2cbcfc06 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:18.760195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c9f60-9e86-4a40-b06e-831e2cbcfc06 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:18.760195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c9f60-9e86-4a40-b06e-831e2cbcfc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:18.805026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1e8c9f60-9e86-4a40-b06e-831e2cbcfc06 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:18.806281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1e8c9f60-9e86-4a40-b06e-831e2cbcfc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:18.982642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0af33cde1b7425181dfacd9ef8fa830', 'request_id': 'req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:19.079262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0af33cde1b7425181dfacd9ef8fa830', 'request_id': 'req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:19.088611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0af33cde1b7425181dfacd9ef8fa830', 'request_id': 'req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:19.093668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 838/1695] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:19.097258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722574ed-7d87-448e-a786-2140bbec0f1a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.097379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722574ed-7d87-448e-a786-2140bbec0f1a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.097619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722574ed-7d87-448e-a786-2140bbec0f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.121886 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] The token cache is being invalidate because project e0af33cde1b7425181dfacd9ef8fa830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:19.122287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:19.123882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ea649c0a-e269-4ac2-9ea7-b0af4b9c9731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:19.129917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 858/1696] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:18 2022] DELETE /identity/v3/projects/e0af33cde1b7425181dfacd9ef8fa830 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:19.131749 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5b14-a859-4eac-a44e-3c5482df11be None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.132636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5b14-a859-4eac-a44e-3c5482df11be None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.132930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5b14-a859-4eac-a44e-3c5482df11be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.141025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-722574ed-7d87-448e-a786-2140bbec0f1a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:19.142266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-722574ed-7d87-448e-a786-2140bbec0f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.256443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a9fb5b14-a859-4eac-a44e-3c5482df11be None None] MFA Rules not processed for user `f8ee605bf3eb44e99d49af0f3a0f578a`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:19.265047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a9fb5b14-a859-4eac-a44e-3c5482df11be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.374464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 839/1697] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:19.375295 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722574ed-7d87-448e-a786-2140bbec0f1a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:19.379420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-722574ed-7d87-448e-a786-2140bbec0f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.465713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 859/1698] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:19.470685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5b14-a859-4eac-a44e-3c5482df11be None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:19.472295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a9fb5b14-a859-4eac-a44e-3c5482df11be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.518188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722574ed-7d87-448e-a786-2140bbec0f1a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:19.529180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1e8bc-1a34-42a4-beab-20a64da6fdfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:19.529953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1e8bc-1a34-42a4-beab-20a64da6fdfa service placement] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.530352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1e8bc-1a34-42a4-beab-20a64da6fdfa service placement] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.530653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1e8bc-1a34-42a4-beab-20a64da6fdfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.533378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-edd1e8bc-1a34-42a4-beab-20a64da6fdfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.549827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1e8bc-1a34-42a4-beab-20a64da6fdfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:19.551977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1e8bc-1a34-42a4-beab-20a64da6fdfa service placement] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:19.552804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5b14-a859-4eac-a44e-3c5482df11be None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:19.558569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0040b0c-0d0e-4fd2-a9dc-6b1fa020ced6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:19.559439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0040b0c-0d0e-4fd2-a9dc-6b1fa020ced6 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.559902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0040b0c-0d0e-4fd2-a9dc-6b1fa020ced6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.560322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0040b0c-0d0e-4fd2-a9dc-6b1fa020ced6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.562830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b0040b0c-0d0e-4fd2-a9dc-6b1fa020ced6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.569140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 840/1699] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:19.570707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cec6fd-7525-4824-b543-9c3a542becee None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.571080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cec6fd-7525-4824-b543-9c3a542becee None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.571426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cec6fd-7525-4824-b543-9c3a542becee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.578461 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0040b0c-0d0e-4fd2-a9dc-6b1fa020ced6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:19.580482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0040b0c-0d0e-4fd2-a9dc-6b1fa020ced6 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:19.594107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-32cec6fd-7525-4824-b543-9c3a542becee None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:19.595480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 860/1700] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:19.595864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-32cec6fd-7525-4824-b543-9c3a542becee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.597849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd05e2-316b-49cf-b0a5-4fe0c4e8befa None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.598365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd05e2-316b-49cf-b0a5-4fe0c4e8befa None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.598872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd05e2-316b-49cf-b0a5-4fe0c4e8befa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.607574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 841/1701] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:19.608585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cec6fd-7525-4824-b543-9c3a542becee None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:19.620866 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-32cec6fd-7525-4824-b543-9c3a542becee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.642618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-17bd05e2-316b-49cf-b0a5-4fe0c4e8befa None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:19.643320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cec6fd-7525-4824-b543-9c3a542becee None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:19.647195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-17bd05e2-316b-49cf-b0a5-4fe0c4e8befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.651002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0ef1e-608e-41f4-a86a-e7928224e18a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:19.651847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0ef1e-608e-41f4-a86a-e7928224e18a service placement] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.652225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0ef1e-608e-41f4-a86a-e7928224e18a service placement] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.652600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0ef1e-608e-41f4-a86a-e7928224e18a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.662331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 861/1702] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:19.664238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd05e2-316b-49cf-b0a5-4fe0c4e8befa None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:19.667254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-17bd05e2-316b-49cf-b0a5-4fe0c4e8befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.674873 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0ef1e-608e-41f4-a86a-e7928224e18a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:19.676686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0ef1e-608e-41f4-a86a-e7928224e18a service placement] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:19.685569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd05e2-316b-49cf-b0a5-4fe0c4e8befa None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:19.691176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 842/1703] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:19.693109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8468f9-d2b7-4153-a40e-bc8bc0ed2968 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.693969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8468f9-d2b7-4153-a40e-bc8bc0ed2968 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.694404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8468f9-d2b7-4153-a40e-bc8bc0ed2968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.695503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea268054-b830-405f-9a91-da1ccef1521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:19.696452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea268054-b830-405f-9a91-da1ccef1521f admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.696739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea268054-b830-405f-9a91-da1ccef1521f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.700143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea268054-b830-405f-9a91-da1ccef1521f admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.701213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea268054-b830-405f-9a91-da1ccef1521f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:19.706146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea268054-b830-405f-9a91-da1ccef1521f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:19.733317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-fb8468f9-d2b7-4153-a40e-bc8bc0ed2968 None None] MFA Rules not processed for user `41ce5cc7ff504d64aac73e48097a4b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:19.735281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fb8468f9-d2b7-4153-a40e-bc8bc0ed2968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.864393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 843/1704] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:19.864393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9f274-e658-428e-8e29-62120933cc9f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.864393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9f274-e658-428e-8e29-62120933cc9f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.864393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9f274-e658-428e-8e29-62120933cc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.900457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-91f9f274-e658-428e-8e29-62120933cc9f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:19.903346 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 862/1705] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:19 2022] POST /identity/v3/projects => generated 379 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:19.904576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea268054-b830-405f-9a91-da1ccef1521f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:19.905058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-91f9f274-e658-428e-8e29-62120933cc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:19.983930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea268054-b830-405f-9a91-da1ccef1521f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:19.990084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fefc9-0a93-452b-bc07-b3a4e3f7bcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:19.990946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fefc9-0a93-452b-bc07-b3a4e3f7bcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:19.991246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fefc9-0a93-452b-bc07-b3a4e3f7bcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:19.991642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fefc9-0a93-452b-bc07-b3a4e3f7bcb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:19.992481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fefc9-0a93-452b-bc07-b3a4e3f7bcb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:19.992800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fefc9-0a93-452b-bc07-b3a4e3f7bcb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:19.994539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fefc9-0a93-452b-bc07-b3a4e3f7bcb5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:20.031587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 863/1706] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:20.033768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fefc9-0a93-452b-bc07-b3a4e3f7bcb5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:20.071019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 844/1707] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:20.073581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9f274-e658-428e-8e29-62120933cc9f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:20.082628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fefc9-0a93-452b-bc07-b3a4e3f7bcb5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:20.090179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa4d10-4b30-41dc-bb8a-45cb94fc5e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:20.090860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa4d10-4b30-41dc-bb8a-45cb94fc5e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:20.091148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa4d10-4b30-41dc-bb8a-45cb94fc5e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:20.091429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa4d10-4b30-41dc-bb8a-45cb94fc5e7f admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:20.092370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa4d10-4b30-41dc-bb8a-45cb94fc5e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:20.095608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa4d10-4b30-41dc-bb8a-45cb94fc5e7f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:20.098466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9f274-e658-428e-8e29-62120933cc9f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:20.110800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba5c46-eb96-4552-bb7c-9c8176d1d415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:20.111564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba5c46-eb96-4552-bb7c-9c8176d1d415 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:20.111851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba5c46-eb96-4552-bb7c-9c8176d1d415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:20.112124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba5c46-eb96-4552-bb7c-9c8176d1d415 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:20.112907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba5c46-eb96-4552-bb7c-9c8176d1d415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:20.115036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba5c46-eb96-4552-bb7c-9c8176d1d415 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:20.126085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:20.184658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 864/1708] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:20 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:20.189324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa4d10-4b30-41dc-bb8a-45cb94fc5e7f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:20.203252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa4d10-4b30-41dc-bb8a-45cb94fc5e7f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:20.212903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654d509-3b7f-447d-a7af-84ee85ec08a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:20.213258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654d509-3b7f-447d-a7af-84ee85ec08a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:20.213541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654d509-3b7f-447d-a7af-84ee85ec08a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:20.213905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654d509-3b7f-447d-a7af-84ee85ec08a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:20.214944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654d509-3b7f-447d-a7af-84ee85ec08a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:20.215075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 845/1709] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:20 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:20.215401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654d509-3b7f-447d-a7af-84ee85ec08a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:20.216672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba5c46-eb96-4552-bb7c-9c8176d1d415 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:20.217164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654d509-3b7f-447d-a7af-84ee85ec08a7 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:20.221291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c2ba5c46-eb96-4552-bb7c-9c8176d1d415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:20.259026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 865/1710] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:20.259026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654d509-3b7f-447d-a7af-84ee85ec08a7 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:20.282252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654d509-3b7f-447d-a7af-84ee85ec08a7 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:20.384313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b68cf-4336-44e1-8478-83a847ac1a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:20.385008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b68cf-4336-44e1-8478-83a847ac1a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:20.385349 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b68cf-4336-44e1-8478-83a847ac1a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:20.385632 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b68cf-4336-44e1-8478-83a847ac1a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:20.386376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b68cf-4336-44e1-8478-83a847ac1a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:20.388488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b68cf-4336-44e1-8478-83a847ac1a9a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:20.391024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba5c46-eb96-4552-bb7c-9c8176d1d415 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:20.404457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 866/1711] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:20.408906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b68cf-4336-44e1-8478-83a847ac1a9a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:20.410584 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:20.412334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:20.414276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:20.414636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] PATH_INFO: `/v3/users/a07977e47fdc4bb0acdcc82e8fa694dd` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:20.416239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07977e47fdc4bb0acdcc82e8fa694dd)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:20.432605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:20.447739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b68cf-4336-44e1-8478-83a847ac1a9a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:20.467584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477a493-c1dd-497a-bda1-6d46bb456804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:20.468304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477a493-c1dd-497a-bda1-6d46bb456804 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:20.468599 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477a493-c1dd-497a-bda1-6d46bb456804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:20.472494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477a493-c1dd-497a-bda1-6d46bb456804 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:20.472494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477a493-c1dd-497a-bda1-6d46bb456804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:20.472494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477a493-c1dd-497a-bda1-6d46bb456804 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:20.569233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:20.584111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 867/1712] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:20 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:20.586146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477a493-c1dd-497a-bda1-6d46bb456804 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:20.655097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477a493-c1dd-497a-bda1-6d46bb456804 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:20.672565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360e3b9-9b12-4375-a01f-7f73d5494c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:20.673437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360e3b9-9b12-4375-a01f-7f73d5494c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:20.673726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360e3b9-9b12-4375-a01f-7f73d5494c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:20.674000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360e3b9-9b12-4375-a01f-7f73d5494c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:20.675708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360e3b9-9b12-4375-a01f-7f73d5494c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:20.677802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360e3b9-9b12-4375-a01f-7f73d5494c6c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:20.704537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07977e47fdc4bb0acdcc82e8fa694dd'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:20.705396 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07977e47fdc4bb0acdcc82e8fa694dd'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:20.726954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07977e47fdc4bb0acdcc82e8fa694dd'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:20.728464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07977e47fdc4bb0acdcc82e8fa694dd'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:20.759581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07977e47fdc4bb0acdcc82e8fa694dd'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:20.845508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 868/1713] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:20 2022] POST /identity/v3/projects => generated 379 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:20.845508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360e3b9-9b12-4375-a01f-7f73d5494c6c admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:20.852106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6360e3b9-9b12-4375-a01f-7f73d5494c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:20.901785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 846/1714] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:20 2022] DELETE /identity/v3/users/a07977e47fdc4bb0acdcc82e8fa694dd => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:20.902910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:20.907106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:21.012777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360e3b9-9b12-4375-a01f-7f73d5494c6c admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:21.018546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957449e7-7b36-45dd-a3ba-cd445157934a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:21.019471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957449e7-7b36-45dd-a3ba-cd445157934a admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.019799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957449e7-7b36-45dd-a3ba-cd445157934a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.025312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957449e7-7b36-45dd-a3ba-cd445157934a admin admin] PATH_INFO: `/v3/projects/e5d6aedf27974026ba83f32cbaaba29b/users/4c7f552bfad14887bd0660059c06015f/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:21.033363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2b0bf-cf72-41c7-a086-5773b4f9312a admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:21.036819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957449e7-7b36-45dd-a3ba-cd445157934a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d6aedf27974026ba83f32cbaaba29b, user_id=4c7f552bfad14887bd0660059c06015f, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:21.038437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:21.039673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.040033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.040285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957449e7-7b36-45dd-a3ba-cd445157934a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:21.040567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] PATH_INFO: `/v3/projects/f4e9b909eb8244d3a025239ee1e8448e` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:21.041884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e9b909eb8244d3a025239ee1e8448e)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:21.044056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:21.148197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 869/1715] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:20 2022] PUT /identity/v3/projects/e5d6aedf27974026ba83f32cbaaba29b/users/4c7f552bfad14887bd0660059c06015f/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:21.151196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957449e7-7b36-45dd-a3ba-cd445157934a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:21.155593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-957449e7-7b36-45dd-a3ba-cd445157934a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:21.204316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e9b909eb8244d3a025239ee1e8448e', 'request_id': 'req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:21.238542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e9b909eb8244d3a025239ee1e8448e', 'request_id': 'req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:21.261144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e9b909eb8244d3a025239ee1e8448e', 'request_id': 'req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:21.335645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957449e7-7b36-45dd-a3ba-cd445157934a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:21.371204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] The token cache is being invalidate because project f4e9b909eb8244d3a025239ee1e8448e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:21.371790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:21.372548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:21.374505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 847/1716] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:20 2022] DELETE /identity/v3/projects/f4e9b909eb8244d3a025239ee1e8448e => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:21.375365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:21.383008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:21.479733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d09a6-b343-44f4-9559-1b339f5a7d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:21.481820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d09a6-b343-44f4-9559-1b339f5a7d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.482116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d09a6-b343-44f4-9559-1b339f5a7d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.482396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d09a6-b343-44f4-9559-1b339f5a7d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:21.483141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d09a6-b343-44f4-9559-1b339f5a7d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:21.487074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d09a6-b343-44f4-9559-1b339f5a7d1f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:21.510058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 870/1717] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:21.516725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764d98-6f3b-4bce-a92c-6e0de581c17d None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.518030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764d98-6f3b-4bce-a92c-6e0de581c17d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.518364 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764d98-6f3b-4bce-a92c-6e0de581c17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:21.555195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f522f0-02f0-4589-b2b0-28e1e1d848f4 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:21.559636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2a764d98-6f3b-4bce-a92c-6e0de581c17d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:21.561184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2a764d98-6f3b-4bce-a92c-6e0de581c17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:21.564032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70d7f9-744e-4f8d-81cd-bbd5e9d56161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:21.566331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70d7f9-744e-4f8d-81cd-bbd5e9d56161 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.566882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70d7f9-744e-4f8d-81cd-bbd5e9d56161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.567374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70d7f9-744e-4f8d-81cd-bbd5e9d56161 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:21.568327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70d7f9-744e-4f8d-81cd-bbd5e9d56161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:21.571032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70d7f9-744e-4f8d-81cd-bbd5e9d56161 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:21.577057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 871/1718] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:21.578154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764d98-6f3b-4bce-a92c-6e0de581c17d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:21.587752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2a764d98-6f3b-4bce-a92c-6e0de581c17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:21.603393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764d98-6f3b-4bce-a92c-6e0de581c17d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:21.609420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40abaaf-f5db-4b09-ad1c-9d471e4c99a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:21.610241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40abaaf-f5db-4b09-ad1c-9d471e4c99a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.610591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40abaaf-f5db-4b09-ad1c-9d471e4c99a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.611030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40abaaf-f5db-4b09-ad1c-9d471e4c99a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:21.612330 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40abaaf-f5db-4b09-ad1c-9d471e4c99a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:21.613695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40abaaf-f5db-4b09-ad1c-9d471e4c99a0 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:21.641789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 872/1719] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:21.650895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40abaaf-f5db-4b09-ad1c-9d471e4c99a0 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:21.652983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a40abaaf-f5db-4b09-ad1c-9d471e4c99a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:21.674664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 848/1720] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:21 2022] POST /identity/v3/users => generated 290 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:21.675116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40abaaf-f5db-4b09-ad1c-9d471e4c99a0 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:21.678017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70d7f9-744e-4f8d-81cd-bbd5e9d56161 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:21.685305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-df70d7f9-744e-4f8d-81cd-bbd5e9d56161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:21.688897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:21.691117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.691710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.692167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] PATH_INFO: `/v3/users/55fd6ab0b4ae4661b0b86133c5fb689a` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:21.694176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55fd6ab0b4ae4661b0b86133c5fb689a)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:21.697004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:21.715719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70d7f9-744e-4f8d-81cd-bbd5e9d56161 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:21.725647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1f768-80ea-42a5-9c62-1caa0e615f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:21.728293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1f768-80ea-42a5-9c62-1caa0e615f5e admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.728582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1f768-80ea-42a5-9c62-1caa0e615f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.728872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1f768-80ea-42a5-9c62-1caa0e615f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:21.732947 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1f768-80ea-42a5-9c62-1caa0e615f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:21.734851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1f768-80ea-42a5-9c62-1caa0e615f5e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:21.750969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 849/1721] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:21.751979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1f768-80ea-42a5-9c62-1caa0e615f5e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:21.756710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-cbe1f768-80ea-42a5-9c62-1caa0e615f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:21.902516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1f768-80ea-42a5-9c62-1caa0e615f5e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:21.907214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946916dd-84ed-4161-9a79-d0e1f427297e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:21.908189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946916dd-84ed-4161-9a79-d0e1f427297e service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.908644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946916dd-84ed-4161-9a79-d0e1f427297e service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.909076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946916dd-84ed-4161-9a79-d0e1f427297e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:21.925578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946916dd-84ed-4161-9a79-d0e1f427297e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:21.927183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946916dd-84ed-4161-9a79-d0e1f427297e service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:21.940428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 850/1722] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:21.941675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946916dd-84ed-4161-9a79-d0e1f427297e service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:21.979902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946916dd-84ed-4161-9a79-d0e1f427297e service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:21.983412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55fd6ab0b4ae4661b0b86133c5fb689a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:21.991393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebf612-8019-4955-8dc6-e917c965836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:21.992501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebf612-8019-4955-8dc6-e917c965836a admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:21.992789 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebf612-8019-4955-8dc6-e917c965836a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:21.993067 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebf612-8019-4955-8dc6-e917c965836a admin admin] PATH_INFO: `/v3/projects/e5d6aedf27974026ba83f32cbaaba29b/users/4c7f552bfad14887bd0660059c06015f/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:22.006088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebf612-8019-4955-8dc6-e917c965836a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d6aedf27974026ba83f32cbaaba29b, user_id=4c7f552bfad14887bd0660059c06015f, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:22.010219 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebf612-8019-4955-8dc6-e917c965836a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:22.022555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 851/1723] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:21 2022] PUT /identity/v3/projects/e5d6aedf27974026ba83f32cbaaba29b/users/4c7f552bfad14887bd0660059c06015f/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:22.023426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebf612-8019-4955-8dc6-e917c965836a admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:22.079955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55fd6ab0b4ae4661b0b86133c5fb689a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:22.080595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55fd6ab0b4ae4661b0b86133c5fb689a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:22.096311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55fd6ab0b4ae4661b0b86133c5fb689a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:22.097899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55fd6ab0b4ae4661b0b86133c5fb689a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:22.147358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 873/1724] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:21 2022] DELETE /identity/v3/users/55fd6ab0b4ae4661b0b86133c5fb689a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:22.151642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:22.154412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:22.298943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebf612-8019-4955-8dc6-e917c965836a admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:22.356331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e2648-34f4-4d1d-ba41-2c0054791737 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:22.371360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29438-6284-4da6-803e-0a7629b4b1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:22.377478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29438-6284-4da6-803e-0a7629b4b1e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:22.378355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29438-6284-4da6-803e-0a7629b4b1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:22.379301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a29438-6284-4da6-803e-0a7629b4b1e8 admin admin] PATH_INFO: `/v3/projects/81f0d69f219148e4a8b49c2465e25d08/users/43847e34424a4b2a8e9c0e4baa73b4e4/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:22.417726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a29438-6284-4da6-803e-0a7629b4b1e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f0d69f219148e4a8b49c2465e25d08, user_id=43847e34424a4b2a8e9c0e4baa73b4e4, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:22.426103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a29438-6284-4da6-803e-0a7629b4b1e8 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:22.475232 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a83619-01a6-42bf-91cd-40fef71d0ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:22.476483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a83619-01a6-42bf-91cd-40fef71d0ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:22.480505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a83619-01a6-42bf-91cd-40fef71d0ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:22.480800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a83619-01a6-42bf-91cd-40fef71d0ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:22.481888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a83619-01a6-42bf-91cd-40fef71d0ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:22.483582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a83619-01a6-42bf-91cd-40fef71d0ca9 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:22.506287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 852/1725] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:22.510978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9705e9-13f5-4dcb-bc1f-15ffa5d5bc2b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:22.511277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9705e9-13f5-4dcb-bc1f-15ffa5d5bc2b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:22.511570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9705e9-13f5-4dcb-bc1f-15ffa5d5bc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:22.519568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 874/1726] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:22 2022] PUT /identity/v3/projects/81f0d69f219148e4a8b49c2465e25d08/users/43847e34424a4b2a8e9c0e4baa73b4e4/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:22.521802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29438-6284-4da6-803e-0a7629b4b1e8 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:22.525363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:22.527210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-98a29438-6284-4da6-803e-0a7629b4b1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:22.584934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8e9705e9-13f5-4dcb-bc1f-15ffa5d5bc2b None None] MFA Rules not processed for user `4c7f552bfad14887bd0660059c06015f`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:22.586300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8e9705e9-13f5-4dcb-bc1f-15ffa5d5bc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:22.709640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a29438-6284-4da6-803e-0a7629b4b1e8 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:22.717025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76f1ed-d65c-4478-92c7-757b42307e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:22.718219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f1ed-d65c-4478-92c7-757b42307e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:22.718219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f1ed-d65c-4478-92c7-757b42307e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:22.718480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f1ed-d65c-4478-92c7-757b42307e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:22.719196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76f1ed-d65c-4478-92c7-757b42307e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:22.721853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76f1ed-d65c-4478-92c7-757b42307e30 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:22.736278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 875/1727] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:22.737734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce796e-5771-442c-b549-cca21fd404d0 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:22.738030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce796e-5771-442c-b549-cca21fd404d0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:22.738307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce796e-5771-442c-b549-cca21fd404d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:22.774733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 853/1728] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:22.775784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9705e9-13f5-4dcb-bc1f-15ffa5d5bc2b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:22.790251 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e8ce796e-5771-442c-b549-cca21fd404d0 None None] MFA Rules not processed for user `43847e34424a4b2a8e9c0e4baa73b4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:22.792547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e8ce796e-5771-442c-b549-cca21fd404d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:22.805811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9705e9-13f5-4dcb-bc1f-15ffa5d5bc2b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:22.816000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6b636-35ec-4318-b9f8-adfcb5861773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:22.816933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6b636-35ec-4318-b9f8-adfcb5861773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:22.817219 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6b636-35ec-4318-b9f8-adfcb5861773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:22.820808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6b636-35ec-4318-b9f8-adfcb5861773 admin admin] PATH_INFO: `/v3/projects/f07300134844484a950ecf052af5ecf7/users/3e018353a3b740a99efcf47b6f59b909/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:22.852848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6b636-35ec-4318-b9f8-adfcb5861773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07300134844484a950ecf052af5ecf7, user_id=3e018353a3b740a99efcf47b6f59b909, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:22.855746 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6b636-35ec-4318-b9f8-adfcb5861773 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:22.906073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 876/1729] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:22.906073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce796e-5771-442c-b549-cca21fd404d0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:22.907891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e8ce796e-5771-442c-b549-cca21fd404d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:22.962661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 854/1730] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:22 2022] PUT /identity/v3/projects/f07300134844484a950ecf052af5ecf7/users/3e018353a3b740a99efcf47b6f59b909/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:22.962661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6b636-35ec-4318-b9f8-adfcb5861773 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:22.978113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fae6b636-35ec-4318-b9f8-adfcb5861773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:23.050508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce796e-5771-442c-b549-cca21fd404d0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:23.120959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6b636-35ec-4318-b9f8-adfcb5861773 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:23.127020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:23.127937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:23.128219 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:23.128494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] PATH_INFO: `/v3/projects/e6d5085e18534f11823c5d1f9375caf4` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:23.129810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d5085e18534f11823c5d1f9375caf4)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:23.132773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:23.159803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:23.216157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2076d4-1cc9-43e1-9f8b-a09b4ec1cab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:23.216833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2076d4-1cc9-43e1-9f8b-a09b4ec1cab1 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:23.217106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2076d4-1cc9-43e1-9f8b-a09b4ec1cab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:23.217385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2076d4-1cc9-43e1-9f8b-a09b4ec1cab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:23.306061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d5085e18534f11823c5d1f9375caf4', 'request_id': 'req-588a9e52-23cb-4c42-881d-b483587ecfbb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.318102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d5085e18534f11823c5d1f9375caf4', 'request_id': 'req-588a9e52-23cb-4c42-881d-b483587ecfbb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.326057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2076d4-1cc9-43e1-9f8b-a09b4ec1cab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:23.327573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2076d4-1cc9-43e1-9f8b-a09b4ec1cab1 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:23.330478 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d5085e18534f11823c5d1f9375caf4', 'request_id': 'req-588a9e52-23cb-4c42-881d-b483587ecfbb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.355896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 877/1731] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:23.356839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2076d4-1cc9-43e1-9f8b-a09b4ec1cab1 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:23.358327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5a2076d4-1cc9-43e1-9f8b-a09b4ec1cab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:23.432123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] The token cache is being invalidate because project e6d5085e18534f11823c5d1f9375caf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:23.432709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.435053 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.436220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 855/1732] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:22 2022] DELETE /identity/v3/projects/e6d5085e18534f11823c5d1f9375caf4 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:23.437404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:23.439200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:23.506438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2076d4-1cc9-43e1-9f8b-a09b4ec1cab1 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:23.573945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a9e52-23cb-4c42-881d-b483587ecfbb admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:23.594706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:23.600372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:23.600372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:23.600372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] PATH_INFO: `/v3/projects/02636e4224fc4543bc39f718a57aa73c` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:23.600372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02636e4224fc4543bc39f718a57aa73c)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:23.601670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:23.660891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35970f07-be43-4fab-ab18-498370f84eb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:23.666876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35970f07-be43-4fab-ab18-498370f84eb6 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:23.666876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35970f07-be43-4fab-ab18-498370f84eb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:23.666876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35970f07-be43-4fab-ab18-498370f84eb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:23.666876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-35970f07-be43-4fab-ab18-498370f84eb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:23.732827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02636e4224fc4543bc39f718a57aa73c', 'request_id': 'req-a54cd381-abef-44ca-8c6a-5dfd800afd86'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.744539 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02636e4224fc4543bc39f718a57aa73c', 'request_id': 'req-a54cd381-abef-44ca-8c6a-5dfd800afd86'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.755672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02636e4224fc4543bc39f718a57aa73c', 'request_id': 'req-a54cd381-abef-44ca-8c6a-5dfd800afd86'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.761996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35970f07-be43-4fab-ab18-498370f84eb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:23.763896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35970f07-be43-4fab-ab18-498370f84eb6 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:23.774529 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 878/1733] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:23.775984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada769f-d4e2-4828-af4c-62a17afac281 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:23.776361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada769f-d4e2-4828-af4c-62a17afac281 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:23.776732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada769f-d4e2-4828-af4c-62a17afac281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:23.804861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-0ada769f-d4e2-4828-af4c-62a17afac281 None None] MFA Rules not processed for user `3e018353a3b740a99efcf47b6f59b909`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:23.806077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-0ada769f-d4e2-4828-af4c-62a17afac281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:23.829410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] The token cache is being invalidate because project 02636e4224fc4543bc39f718a57aa73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:23.829725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.831505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a54cd381-abef-44ca-8c6a-5dfd800afd86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:23.843557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 856/1734] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:23 2022] DELETE /identity/v3/projects/02636e4224fc4543bc39f718a57aa73c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:23.843557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d01e1-3232-4666-b441-c777603eaf33 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:23.843557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d01e1-3232-4666-b441-c777603eaf33 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:23.843557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d01e1-3232-4666-b441-c777603eaf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:23.894391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b76d01e1-3232-4666-b441-c777603eaf33 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:23.895658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b76d01e1-3232-4666-b441-c777603eaf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:23.946227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 879/1735] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:23.947355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08507-c76a-4568-b630-557ee86d501f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:23.948482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08507-c76a-4568-b630-557ee86d501f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:23.948765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08507-c76a-4568-b630-557ee86d501f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:23.979786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-99f08507-c76a-4568-b630-557ee86d501f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:23.980406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-99f08507-c76a-4568-b630-557ee86d501f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:23.990961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 880/1736] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:23.991860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f08507-c76a-4568-b630-557ee86d501f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:23.999586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 857/1737] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:24.004997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-99f08507-c76a-4568-b630-557ee86d501f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:24.009606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb77c2-3971-4b2e-8b03-bfde27031440 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.010368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb77c2-3971-4b2e-8b03-bfde27031440 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.010727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb77c2-3971-4b2e-8b03-bfde27031440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.034082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:24.041728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-6ebb77c2-3971-4b2e-8b03-bfde27031440 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:24.043006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6ebb77c2-3971-4b2e-8b03-bfde27031440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:24.052781 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f08507-c76a-4568-b630-557ee86d501f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:24.053996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 858/1738] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:24.055187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360d247-3168-4fb4-b5eb-a44e6c1c0c68 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.055489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360d247-3168-4fb4-b5eb-a44e6c1c0c68 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.055797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360d247-3168-4fb4-b5eb-a44e6c1c0c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.063494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d4c63-01ec-4a8a-a565-aa7d7af4de81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:24.064209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d4c63-01ec-4a8a-a565-aa7d7af4de81 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.072954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d4c63-01ec-4a8a-a565-aa7d7af4de81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.072954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d4c63-01ec-4a8a-a565-aa7d7af4de81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.072954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d4c63-01ec-4a8a-a565-aa7d7af4de81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:24.072954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d4c63-01ec-4a8a-a565-aa7d7af4de81 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:24.075105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7360d247-3168-4fb4-b5eb-a44e6c1c0c68 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:24.076362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7360d247-3168-4fb4-b5eb-a44e6c1c0c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:24.084678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 859/1739] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:24.086005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360d247-3168-4fb4-b5eb-a44e6c1c0c68 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:24.095456 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360d247-3168-4fb4-b5eb-a44e6c1c0c68 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:24.100105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e5de4-a96c-4428-b134-49e4a119aee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:24.100866 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e5de4-a96c-4428-b134-49e4a119aee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.101155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e5de4-a96c-4428-b134-49e4a119aee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.101458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e5de4-a96c-4428-b134-49e4a119aee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.102205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e5de4-a96c-4428-b134-49e4a119aee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:24.102519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e5de4-a96c-4428-b134-49e4a119aee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:24.104207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e5de4-a96c-4428-b134-49e4a119aee6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:24.115734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 860/1740] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:24.116574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e5de4-a96c-4428-b134-49e4a119aee6 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:24.127670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e5de4-a96c-4428-b134-49e4a119aee6 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:24.133178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193b79b-28a7-46f4-acf0-260d8156f002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:24.133859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193b79b-28a7-46f4-acf0-260d8156f002 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.134149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193b79b-28a7-46f4-acf0-260d8156f002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.134510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193b79b-28a7-46f4-acf0-260d8156f002 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.135914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193b79b-28a7-46f4-acf0-260d8156f002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:24.142823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193b79b-28a7-46f4-acf0-260d8156f002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:24.144608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193b79b-28a7-46f4-acf0-260d8156f002 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:24.158115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 861/1741] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:24.158964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193b79b-28a7-46f4-acf0-260d8156f002 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:24.172382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193b79b-28a7-46f4-acf0-260d8156f002 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:24.177324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b8866-e267-4d9c-ad1b-df935de60121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:24.178011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b8866-e267-4d9c-ad1b-df935de60121 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.178330 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b8866-e267-4d9c-ad1b-df935de60121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.178635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b8866-e267-4d9c-ad1b-df935de60121 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.179401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b8866-e267-4d9c-ad1b-df935de60121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:24.181441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b8866-e267-4d9c-ad1b-df935de60121 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:24.242376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 881/1742] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:23 2022] POST /identity/v3/projects => generated 381 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:24.242376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d4c63-01ec-4a8a-a565-aa7d7af4de81 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:24.245322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8a3d4c63-01ec-4a8a-a565-aa7d7af4de81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:24.292103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 862/1743] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:24 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:24.293048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b8866-e267-4d9c-ad1b-df935de60121 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:24.433001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:24.438738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d4c63-01ec-4a8a-a565-aa7d7af4de81 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:24.446221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:24.447358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.447685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.447965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] PATH_INFO: `/v3/users/f8ee605bf3eb44e99d49af0f3a0f578a` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.449250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ee605bf3eb44e99d49af0f3a0f578a)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:24.452296 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:24.460549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b8866-e267-4d9c-ad1b-df935de60121 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:24.490714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f60630-b652-4a80-9438-31cbc37d0cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:24.490714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f60630-b652-4a80-9438-31cbc37d0cea admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.490714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f60630-b652-4a80-9438-31cbc37d0cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.490714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f60630-b652-4a80-9438-31cbc37d0cea admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.493464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f60630-b652-4a80-9438-31cbc37d0cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:24.495670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f60630-b652-4a80-9438-31cbc37d0cea admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:24.608211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 863/1744] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:24 2022] POST /identity/v3/projects => generated 369 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:24.609301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f60630-b652-4a80-9438-31cbc37d0cea admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:24.685425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f60630-b652-4a80-9438-31cbc37d0cea admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:24.692927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff1416-3234-47ff-8178-2d92d2b12749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:24.693634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff1416-3234-47ff-8178-2d92d2b12749 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.693917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff1416-3234-47ff-8178-2d92d2b12749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.694191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff1416-3234-47ff-8178-2d92d2b12749 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.695497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff1416-3234-47ff-8178-2d92d2b12749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:24.697438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff1416-3234-47ff-8178-2d92d2b12749 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:24.733206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ee605bf3eb44e99d49af0f3a0f578a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:24.807448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 864/1745] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:24 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:24.808314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff1416-3234-47ff-8178-2d92d2b12749 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:24.827649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ee605bf3eb44e99d49af0f3a0f578a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:24.829190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ee605bf3eb44e99d49af0f3a0f578a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:24.829305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff1416-3234-47ff-8178-2d92d2b12749 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:24.838677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ee605bf3eb44e99d49af0f3a0f578a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:24.840328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ee605bf3eb44e99d49af0f3a0f578a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:24.846639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92aba96-a378-4a41-9ae5-0c03ece50cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:24.847637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92aba96-a378-4a41-9ae5-0c03ece50cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.847637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92aba96-a378-4a41-9ae5-0c03ece50cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.847893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92aba96-a378-4a41-9ae5-0c03ece50cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.848601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92aba96-a378-4a41-9ae5-0c03ece50cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:24.851486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92aba96-a378-4a41-9ae5-0c03ece50cb8 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:24.864013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 882/1746] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:24 2022] DELETE /identity/v3/users/f8ee605bf3eb44e99d49af0f3a0f578a => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:24.868493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:24.891429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:24.983939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cbfac-bef3-49f1-8d86-ef55fafd7f12 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:24.990629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2a769-3083-4004-86cd-5e31a59f2572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:24.991649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2a769-3083-4004-86cd-5e31a59f2572 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:24.992374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2a769-3083-4004-86cd-5e31a59f2572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:24.992825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2a769-3083-4004-86cd-5e31a59f2572 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:24.993838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2a769-3083-4004-86cd-5e31a59f2572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:24.996238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2a769-3083-4004-86cd-5e31a59f2572 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.003843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 865/1747] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:24 2022] POST /identity/v3/users => generated 294 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:25.004813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92aba96-a378-4a41-9ae5-0c03ece50cb8 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.006639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d92aba96-a378-4a41-9ae5-0c03ece50cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:25.036840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:25.044128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92aba96-a378-4a41-9ae5-0c03ece50cb8 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.083223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56fbea-426f-4052-bf84-79703ca0f4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.088652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56fbea-426f-4052-bf84-79703ca0f4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.092247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56fbea-426f-4052-bf84-79703ca0f4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.093112 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56fbea-426f-4052-bf84-79703ca0f4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.096989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f56fbea-426f-4052-bf84-79703ca0f4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.098655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f56fbea-426f-4052-bf84-79703ca0f4b4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.117404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 883/1748] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:24 2022] POST /identity/v3/users => generated 285 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:25.121434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d1db9-1657-463c-9bb8-8c1662e3a95a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.122262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d1db9-1657-463c-9bb8-8c1662e3a95a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.122724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d1db9-1657-463c-9bb8-8c1662e3a95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.142030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 866/1749] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:25.146712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56fbea-426f-4052-bf84-79703ca0f4b4 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.146712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0f56fbea-426f-4052-bf84-79703ca0f4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:25.158410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-879d1db9-1657-463c-9bb8-8c1662e3a95a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:25.159797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-879d1db9-1657-463c-9bb8-8c1662e3a95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:25.159956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56fbea-426f-4052-bf84-79703ca0f4b4 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.166276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a59a4-4d52-4d43-968a-4063ec0d376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.167007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a59a4-4d52-4d43-968a-4063ec0d376b admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.167296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a59a4-4d52-4d43-968a-4063ec0d376b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.167635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a59a4-4d52-4d43-968a-4063ec0d376b admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.168418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a59a4-4d52-4d43-968a-4063ec0d376b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.169625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 884/1750] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:25.170095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a59a4-4d52-4d43-968a-4063ec0d376b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.173682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d1db9-1657-463c-9bb8-8c1662e3a95a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.185410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d1db9-1657-463c-9bb8-8c1662e3a95a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.189059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 867/1751] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:25.189059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a59a4-4d52-4d43-968a-4063ec0d376b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.191382 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50471c19-18fb-4200-b444-3fb48b77d6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.192099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50471c19-18fb-4200-b444-3fb48b77d6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.192603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50471c19-18fb-4200-b444-3fb48b77d6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.192882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50471c19-18fb-4200-b444-3fb48b77d6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.193621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50471c19-18fb-4200-b444-3fb48b77d6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.195504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50471c19-18fb-4200-b444-3fb48b77d6c1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.202218 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a59a4-4d52-4d43-968a-4063ec0d376b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.208068 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedefd61-4121-4cb4-befd-c114d3745da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.213840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 885/1752] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:25.213840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50471c19-18fb-4200-b444-3fb48b77d6c1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.214777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-50471c19-18fb-4200-b444-3fb48b77d6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:25.220660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedefd61-4121-4cb4-befd-c114d3745da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.220952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedefd61-4121-4cb4-befd-c114d3745da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.221280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedefd61-4121-4cb4-befd-c114d3745da1 admin admin] PATH_INFO: `/v3/projects/13324525110c404ca7d9bab534f5e7ca/users/bdbdd503440a4cd29a46a3a56be2bfff/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.249655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:25.259565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedefd61-4121-4cb4-befd-c114d3745da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13324525110c404ca7d9bab534f5e7ca, user_id=bdbdd503440a4cd29a46a3a56be2bfff, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.262938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedefd61-4121-4cb4-befd-c114d3745da1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.322468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50471c19-18fb-4200-b444-3fb48b77d6c1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.327441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99b1fb-72a1-46f8-8b31-405df0988b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.331379 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99b1fb-72a1-46f8-8b31-405df0988b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.331759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99b1fb-72a1-46f8-8b31-405df0988b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.332211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99b1fb-72a1-46f8-8b31-405df0988b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.350226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99b1fb-72a1-46f8-8b31-405df0988b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.352697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99b1fb-72a1-46f8-8b31-405df0988b74 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.364847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 886/1753] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:25.365660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99b1fb-72a1-46f8-8b31-405df0988b74 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.375889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99b1fb-72a1-46f8-8b31-405df0988b74 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.380960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2eefb-0b6d-48ee-bf0e-22e3f16d4063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.381862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2eefb-0b6d-48ee-bf0e-22e3f16d4063 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.382181 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2eefb-0b6d-48ee-bf0e-22e3f16d4063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.382440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2eefb-0b6d-48ee-bf0e-22e3f16d4063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.383179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2eefb-0b6d-48ee-bf0e-22e3f16d4063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.384937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2eefb-0b6d-48ee-bf0e-22e3f16d4063 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.389051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 868/1754] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:25 2022] PUT /identity/v3/projects/13324525110c404ca7d9bab534f5e7ca/users/bdbdd503440a4cd29a46a3a56be2bfff/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:25.395958 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedefd61-4121-4cb4-befd-c114d3745da1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.401036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 887/1755] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:25.402441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cdc9b-f2e1-4157-b7c8-7dfcc7de588d None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.402737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cdc9b-f2e1-4157-b7c8-7dfcc7de588d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.403014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cdc9b-f2e1-4157-b7c8-7dfcc7de588d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.430382 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-519cdc9b-f2e1-4157-b7c8-7dfcc7de588d None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:25.432841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-519cdc9b-f2e1-4157-b7c8-7dfcc7de588d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:25.508476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 888/1756] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:25.509703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cdc9b-f2e1-4157-b7c8-7dfcc7de588d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.597153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cdc9b-f2e1-4157-b7c8-7dfcc7de588d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.599966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedefd61-4121-4cb4-befd-c114d3745da1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.603364 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598b829-6e91-4981-987f-530d93f91349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.604299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598b829-6e91-4981-987f-530d93f91349 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.605402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598b829-6e91-4981-987f-530d93f91349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.606010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598b829-6e91-4981-987f-530d93f91349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.607462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598b829-6e91-4981-987f-530d93f91349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.609612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598b829-6e91-4981-987f-530d93f91349 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.621241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5326c3-596f-4909-add3-be40af9a60ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.622202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5326c3-596f-4909-add3-be40af9a60ad admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.622636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5326c3-596f-4909-add3-be40af9a60ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.623090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5326c3-596f-4909-add3-be40af9a60ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.624105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5326c3-596f-4909-add3-be40af9a60ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.625985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5326c3-596f-4909-add3-be40af9a60ad admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.626543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 889/1757] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:25.627803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598b829-6e91-4981-987f-530d93f91349 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.646683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598b829-6e91-4981-987f-530d93f91349 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.653913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37733bb-a626-4418-a68e-0c3ab5291d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.655268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37733bb-a626-4418-a68e-0c3ab5291d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.655818 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37733bb-a626-4418-a68e-0c3ab5291d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.656275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37733bb-a626-4418-a68e-0c3ab5291d48 admin admin] PATH_INFO: `/v3/projects/799a3b48da7542b1bc77727a36f5568c/users/2de6f034e5fc488fa6e10ab86b8bf271/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.662517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 869/1758] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:25.663439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5326c3-596f-4909-add3-be40af9a60ad admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.685372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5326c3-596f-4909-add3-be40af9a60ad admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:25.686786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37733bb-a626-4418-a68e-0c3ab5291d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=799a3b48da7542b1bc77727a36f5568c, user_id=2de6f034e5fc488fa6e10ab86b8bf271, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.695996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37733bb-a626-4418-a68e-0c3ab5291d48 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.698558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735c121-5c07-4c2d-a5ed-b98a728ce364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:25.699665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735c121-5c07-4c2d-a5ed-b98a728ce364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.699954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735c121-5c07-4c2d-a5ed-b98a728ce364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.700238 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735c121-5c07-4c2d-a5ed-b98a728ce364 admin admin] PATH_INFO: `/v3/projects/13324525110c404ca7d9bab534f5e7ca/users/bdbdd503440a4cd29a46a3a56be2bfff/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.711956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735c121-5c07-4c2d-a5ed-b98a728ce364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13324525110c404ca7d9bab534f5e7ca, user_id=bdbdd503440a4cd29a46a3a56be2bfff, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:25.714940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735c121-5c07-4c2d-a5ed-b98a728ce364 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:25.728517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 870/1759] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:25 2022] PUT /identity/v3/projects/13324525110c404ca7d9bab534f5e7ca/users/bdbdd503440a4cd29a46a3a56be2bfff/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:25.729376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735c121-5c07-4c2d-a5ed-b98a728ce364 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:25.790340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 890/1760] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:25 2022] PUT /identity/v3/projects/799a3b48da7542b1bc77727a36f5568c/users/2de6f034e5fc488fa6e10ab86b8bf271/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:25.791774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfeddbd-5b8f-457e-a816-ad68dfde02bb None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:25.792073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfeddbd-5b8f-457e-a816-ad68dfde02bb None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:25.792356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfeddbd-5b8f-457e-a816-ad68dfde02bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:25.869744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-6dfeddbd-5b8f-457e-a816-ad68dfde02bb None None] MFA Rules not processed for user `bdbdd503440a4cd29a46a3a56be2bfff`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:25.871385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6dfeddbd-5b8f-457e-a816-ad68dfde02bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:25.886124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735c121-5c07-4c2d-a5ed-b98a728ce364 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:26.034274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e7aa3-fcf4-4d51-b798-c2c3ddfb2b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:26.035900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:26.036999 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e7aa3-fcf4-4d51-b798-c2c3ddfb2b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:26.037350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e7aa3-fcf4-4d51-b798-c2c3ddfb2b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:26.037627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e7aa3-fcf4-4d51-b798-c2c3ddfb2b35 admin admin] PATH_INFO: `/v3/projects/4203bbdff4f14d9abb26c0fb343e1f73/users/0327d85c4b364cceb5db7f5afab6019f/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:26.054873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 891/1761] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:26.059398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e7aa3-fcf4-4d51-b798-c2c3ddfb2b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4203bbdff4f14d9abb26c0fb343e1f73, user_id=0327d85c4b364cceb5db7f5afab6019f, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:26.061189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020f0a2-8410-401f-8e79-dbab20c6ae8e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:26.061300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e7aa3-fcf4-4d51-b798-c2c3ddfb2b35 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:26.062126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020f0a2-8410-401f-8e79-dbab20c6ae8e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:26.062610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020f0a2-8410-401f-8e79-dbab20c6ae8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:26.101297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-f020f0a2-8410-401f-8e79-dbab20c6ae8e None None] MFA Rules not processed for user `2de6f034e5fc488fa6e10ab86b8bf271`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:26.102941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f020f0a2-8410-401f-8e79-dbab20c6ae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:26.155358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 871/1762] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:25 2022] PUT /identity/v3/projects/4203bbdff4f14d9abb26c0fb343e1f73/users/0327d85c4b364cceb5db7f5afab6019f/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:26.156352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e7aa3-fcf4-4d51-b798-c2c3ddfb2b35 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:26.161229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-db2e7aa3-fcf4-4d51-b798-c2c3ddfb2b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:26.228371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:26.230774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 892/1763] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:26.235181 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020f0a2-8410-401f-8e79-dbab20c6ae8e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:26.236731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f020f0a2-8410-401f-8e79-dbab20c6ae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:26.304261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e7aa3-fcf4-4d51-b798-c2c3ddfb2b35 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:26.311046 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406f764-b55b-4f63-8472-b76b259bc118 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:26.311806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f764-b55b-4f63-8472-b76b259bc118 service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:26.312102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f764-b55b-4f63-8472-b76b259bc118 service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:26.312387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f764-b55b-4f63-8472-b76b259bc118 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:26.366611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020f0a2-8410-401f-8e79-dbab20c6ae8e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:26.396143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406f764-b55b-4f63-8472-b76b259bc118 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:26.399331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406f764-b55b-4f63-8472-b76b259bc118 service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:26.410069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 872/1764] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:36:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:36:26.412848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc3a28-db62-4d63-a5c1-699971fb68f8 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:26.414047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc3a28-db62-4d63-a5c1-699971fb68f8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:26.414526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc3a28-db62-4d63-a5c1-699971fb68f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:26.472995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-75fc3a28-db62-4d63-a5c1-699971fb68f8 None None] MFA Rules not processed for user `0327d85c4b364cceb5db7f5afab6019f`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:26.474658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-75fc3a28-db62-4d63-a5c1-699971fb68f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:26.484239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7adfe-b682-4c99-b40e-b99de95672db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:26.487196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7adfe-b682-4c99-b40e-b99de95672db service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:26.487505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7adfe-b682-4c99-b40e-b99de95672db service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:26.488145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7adfe-b682-4c99-b40e-b99de95672db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:26.619168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 873/1765] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:26.621246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc3a28-db62-4d63-a5c1-699971fb68f8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:26.698735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7adfe-b682-4c99-b40e-b99de95672db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:26.703165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7adfe-b682-4c99-b40e-b99de95672db service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:26.743909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc3a28-db62-4d63-a5c1-699971fb68f8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:26.746901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 893/1766] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:26.747791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7adfe-b682-4c99-b40e-b99de95672db service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:26.767389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fdbf6-e3f9-49f2-a007-a2f8ce06a6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:26.770384 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fdbf6-e3f9-49f2-a007-a2f8ce06a6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:26.770861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fdbf6-e3f9-49f2-a007-a2f8ce06a6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:26.771189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fdbf6-e3f9-49f2-a007-a2f8ce06a6c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:26.774344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fdbf6-e3f9-49f2-a007-a2f8ce06a6c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:26.779475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fdbf6-e3f9-49f2-a007-a2f8ce06a6c7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:26.784136 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7adfe-b682-4c99-b40e-b99de95672db service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:26.807882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239827f4-781b-45cd-ba07-14786f37d62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:26.810324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239827f4-781b-45cd-ba07-14786f37d62b admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:26.811631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239827f4-781b-45cd-ba07-14786f37d62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:26.811989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239827f4-781b-45cd-ba07-14786f37d62b admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:26.821657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239827f4-781b-45cd-ba07-14786f37d62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:26.825713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239827f4-781b-45cd-ba07-14786f37d62b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:26.921836 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 874/1767] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:26 2022] POST /identity/v3/projects => generated 387 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:26.925650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fdbf6-e3f9-49f2-a007-a2f8ce06a6c7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:26.932596 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-630fdbf6-e3f9-49f2-a007-a2f8ce06a6c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:26.993096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 894/1768] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:26 2022] POST /identity/v3/projects => generated 369 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:26.994581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239827f4-781b-45cd-ba07-14786f37d62b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:27.098438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fdbf6-e3f9-49f2-a007-a2f8ce06a6c7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:27.175322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239827f4-781b-45cd-ba07-14786f37d62b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:27.196248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872e291-e004-44cd-949b-4dd80ecd3b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:27.200997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872e291-e004-44cd-949b-4dd80ecd3b3e admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:27.201292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872e291-e004-44cd-949b-4dd80ecd3b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:27.201578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872e291-e004-44cd-949b-4dd80ecd3b3e admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:27.202314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872e291-e004-44cd-949b-4dd80ecd3b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:27.211595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872e291-e004-44cd-949b-4dd80ecd3b3e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:27.245663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:27.248786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:27.249205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:27.249482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] PATH_INFO: `/v3/projects/aa7a1e861f514e8cbe29db155db43981` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:27.252779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa7a1e861f514e8cbe29db155db43981)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:27.256426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:27.269418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:27.394680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 895/1769] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:26 2022] POST /identity/v3/users => generated 300 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:27.395986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872e291-e004-44cd-949b-4dd80ecd3b3e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:27.446835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872e291-e004-44cd-949b-4dd80ecd3b3e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:27.479954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa7a1e861f514e8cbe29db155db43981', 'request_id': 'req-65252e6a-32fa-4d42-bfc1-1167a4f632d6'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:27.485641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:27.508863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa7a1e861f514e8cbe29db155db43981', 'request_id': 'req-65252e6a-32fa-4d42-bfc1-1167a4f632d6'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:27.537072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa7a1e861f514e8cbe29db155db43981', 'request_id': 'req-65252e6a-32fa-4d42-bfc1-1167a4f632d6'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:27.773813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56fe75-e2d6-4e1f-a700-072e8c08a28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:27.773813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56fe75-e2d6-4e1f-a700-072e8c08a28b admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:27.773813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56fe75-e2d6-4e1f-a700-072e8c08a28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:27.773813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56fe75-e2d6-4e1f-a700-072e8c08a28b admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:27.778058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56fe75-e2d6-4e1f-a700-072e8c08a28b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:27.778058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56fe75-e2d6-4e1f-a700-072e8c08a28b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:28.156636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] The token cache is being invalidate because project aa7a1e861f514e8cbe29db155db43981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:28.156636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.156636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.156636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 875/1770] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:26 2022] DELETE /identity/v3/projects/aa7a1e861f514e8cbe29db155db43981 => generated 0 bytes in 1228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:28.156636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:28.157846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:28.225415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 896/1771] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:27 2022] POST /identity/v3/users => generated 291 bytes in 827 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:28.225415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e5001-693d-4293-87b0-8038f818a7be None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:28.225415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e5001-693d-4293-87b0-8038f818a7be None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:28.227368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e5001-693d-4293-87b0-8038f818a7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:28.251185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65252e6a-32fa-4d42-bfc1-1167a4f632d6 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:28.259886 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c93e5001-693d-4293-87b0-8038f818a7be None None] MFA Rules not processed for user `2b87f03cae6c4ca98a82dd5d981ed224`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:28.261450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c93e5001-693d-4293-87b0-8038f818a7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:28.265256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:28.266074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:28.266314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:28.266661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:28.276318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:28.357250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 897/1772] 10.222.0.55 () {58 vars in 1015 bytes} [Mon Jun 27 09:36:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:28.358135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e5001-693d-4293-87b0-8038f818a7be None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:28.360423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c93e5001-693d-4293-87b0-8038f818a7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:28.382785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:28.385604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:28.394307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e5001-693d-4293-87b0-8038f818a7be None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:28.399233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 876/1773] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:28.400234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:28.401714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:28.407574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cdffd-6128-42ae-82b8-32d299fbe748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:28.408329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cdffd-6128-42ae-82b8-32d299fbe748 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:28.408612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cdffd-6128-42ae-82b8-32d299fbe748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:28.408884 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cdffd-6128-42ae-82b8-32d299fbe748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:28.413958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cdffd-6128-42ae-82b8-32d299fbe748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:28.415971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cdffd-6128-42ae-82b8-32d299fbe748 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:28.418203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724829-564b-4313-8547-a6b23aa53c79 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:28.425005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:28.425906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:28.426254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:28.426556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] PATH_INFO: `/v3/users/43847e34424a4b2a8e9c0e4baa73b4e4` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:28.428067 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] RBAC: Authorizing `identity:delete_user(user_id=43847e34424a4b2a8e9c0e4baa73b4e4)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:28.430090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:28.441831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 898/1774] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:28.443165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cdffd-6128-42ae-82b8-32d299fbe748 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:28.446929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fc1cdffd-6128-42ae-82b8-32d299fbe748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:28.484334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cdffd-6128-42ae-82b8-32d299fbe748 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:28.498106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:28.498106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:28.498106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:28.498106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] PATH_INFO: `/v3/users/4c7f552bfad14887bd0660059c06015f` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:28.498448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7f552bfad14887bd0660059c06015f)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:28.500565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:28.541291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:28.617656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43847e34424a4b2a8e9c0e4baa73b4e4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.618472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43847e34424a4b2a8e9c0e4baa73b4e4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.623080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:28.639017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43847e34424a4b2a8e9c0e4baa73b4e4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.640023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43847e34424a4b2a8e9c0e4baa73b4e4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.678153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43847e34424a4b2a8e9c0e4baa73b4e4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.730597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7f552bfad14887bd0660059c06015f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.743467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 877/1775] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:28 2022] DELETE /identity/v3/users/43847e34424a4b2a8e9c0e4baa73b4e4 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:28.744544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:28.745997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:28.794158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7f552bfad14887bd0660059c06015f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.812298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7f552bfad14887bd0660059c06015f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.812298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7f552bfad14887bd0660059c06015f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.812298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7f552bfad14887bd0660059c06015f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:28.860013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 899/1776] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:28 2022] DELETE /identity/v3/users/4c7f552bfad14887bd0660059c06015f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:28.861188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:28.865115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:28.869139 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:28.892112 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fa8a5-ba8f-48d0-87cc-93cdbfe4727b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:28.984911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21525e-8c77-409a-b88b-c6054370023b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:28.998314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8811ed-9544-464b-83b7-98c717696924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:28.999480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8811ed-9544-464b-83b7-98c717696924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:28.999969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8811ed-9544-464b-83b7-98c717696924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.000591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8811ed-9544-464b-83b7-98c717696924 admin admin] PATH_INFO: `/v3/projects/ee21170b05464f4a93799cc486c6f18f/users/b55ee5bbfaec4a319d957049712d0606/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.001960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc36be2-6c57-496b-9de0-1f5f4afa2c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:29.003026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc36be2-6c57-496b-9de0-1f5f4afa2c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.004297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc36be2-6c57-496b-9de0-1f5f4afa2c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.015113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc36be2-6c57-496b-9de0-1f5f4afa2c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.015113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc36be2-6c57-496b-9de0-1f5f4afa2c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.015113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc36be2-6c57-496b-9de0-1f5f4afa2c97 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.031239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 878/1777] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:29.036617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc36be2-6c57-496b-9de0-1f5f4afa2c97 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:29.036748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8811ed-9544-464b-83b7-98c717696924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee21170b05464f4a93799cc486c6f18f, user_id=b55ee5bbfaec4a319d957049712d0606, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.036865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8dc36be2-6c57-496b-9de0-1f5f4afa2c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:29.040992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8811ed-9544-464b-83b7-98c717696924 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.134302 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc36be2-6c57-496b-9de0-1f5f4afa2c97 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:29.152343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 900/1778] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:28 2022] PUT /identity/v3/projects/ee21170b05464f4a93799cc486c6f18f/users/b55ee5bbfaec4a319d957049712d0606/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:29.156221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8811ed-9544-464b-83b7-98c717696924 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:29.167119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:29.167933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.168180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.168461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.170565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:29.188402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.190145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.200641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 879/1779] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:29.201596 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:29.203132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:29.222529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9634f-5e54-42f4-884d-86367709c4ac service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:29.241512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8811ed-9544-464b-83b7-98c717696924 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:29.246872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd6b15-a81c-4cd3-a343-075529aa4c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:29.247716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd6b15-a81c-4cd3-a343-075529aa4c36 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.248105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd6b15-a81c-4cd3-a343-075529aa4c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.248539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd6b15-a81c-4cd3-a343-075529aa4c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.280213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd6b15-a81c-4cd3-a343-075529aa4c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.282501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd6b15-a81c-4cd3-a343-075529aa4c36 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.309503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 901/1780] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:29.311682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd6b15-a81c-4cd3-a343-075529aa4c36 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:29.333467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd6b15-a81c-4cd3-a343-075529aa4c36 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:29.337461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f9746-a51a-4f4d-b787-b63449673afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:29.338287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f9746-a51a-4f4d-b787-b63449673afe service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.338554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f9746-a51a-4f4d-b787-b63449673afe service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.339100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f9746-a51a-4f4d-b787-b63449673afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.339331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f324b-2fbd-48ac-bc84-a4658f13df70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:29.340125 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f324b-2fbd-48ac-bc84-a4658f13df70 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.340409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f324b-2fbd-48ac-bc84-a4658f13df70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.340800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f324b-2fbd-48ac-bc84-a4658f13df70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.349710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f324b-2fbd-48ac-bc84-a4658f13df70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.352339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f324b-2fbd-48ac-bc84-a4658f13df70 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.363335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 902/1781] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:29.375507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f9746-a51a-4f4d-b787-b63449673afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.377174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2f28a-f0d4-44ec-af91-bb66fbc8a7ab None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.377293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f9746-a51a-4f4d-b787-b63449673afe service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.378116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2f28a-f0d4-44ec-af91-bb66fbc8a7ab None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.378406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2f28a-f0d4-44ec-af91-bb66fbc8a7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.408940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 880/1782] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:29.410865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f9746-a51a-4f4d-b787-b63449673afe service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:29.413863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e35f9746-a51a-4f4d-b787-b63449673afe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:29.419711 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d3e2f28a-f0d4-44ec-af91-bb66fbc8a7ab None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:29.420949 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d3e2f28a-f0d4-44ec-af91-bb66fbc8a7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:29.612596 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f9746-a51a-4f4d-b787-b63449673afe service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:29.614331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 903/1783] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:29.615345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2f28a-f0d4-44ec-af91-bb66fbc8a7ab None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:29.644169 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2f28a-f0d4-44ec-af91-bb66fbc8a7ab None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:29.647882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:29.649656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404e7f3-a290-43f1-a8cb-f3aafba06a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:29.657065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a17f4-85f3-4b12-a9b7-b40101f470b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:29.657769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a17f4-85f3-4b12-a9b7-b40101f470b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.658058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a17f4-85f3-4b12-a9b7-b40101f470b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.658337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a17f4-85f3-4b12-a9b7-b40101f470b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.659085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a17f4-85f3-4b12-a9b7-b40101f470b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.659257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404e7f3-a290-43f1-a8cb-f3aafba06a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.660526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404e7f3-a290-43f1-a8cb-f3aafba06a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.660884 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a17f4-85f3-4b12-a9b7-b40101f470b4 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.661980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404e7f3-a290-43f1-a8cb-f3aafba06a64 admin admin] PATH_INFO: `/v3/projects/09380167a6fd4cdea1334201b55c9537/users/d45038db2af04560b48b2156a7cc77a4/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.688382 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 904/1784] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:29.688382 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a17f4-85f3-4b12-a9b7-b40101f470b4 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:29.710411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a17f4-85f3-4b12-a9b7-b40101f470b4 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:29.719428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c24865-f84f-4f62-8e56-473244d20d1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:29.720134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c24865-f84f-4f62-8e56-473244d20d1e service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.720423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c24865-f84f-4f62-8e56-473244d20d1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.720708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c24865-f84f-4f62-8e56-473244d20d1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.740216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:29.743025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404e7f3-a290-43f1-a8cb-f3aafba06a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09380167a6fd4cdea1334201b55c9537, user_id=d45038db2af04560b48b2156a7cc77a4, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.751309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c24865-f84f-4f62-8e56-473244d20d1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.757441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c24865-f84f-4f62-8e56-473244d20d1e service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.757555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404e7f3-a290-43f1-a8cb-f3aafba06a64 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.788381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 905/1785] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:29.796811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c24865-f84f-4f62-8e56-473244d20d1e service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:29.826237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c24865-f84f-4f62-8e56-473244d20d1e service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:29.835268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7b3b3-1114-46ea-b4b8-ce3411e553c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:29.838375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b3b3-1114-46ea-b4b8-ce3411e553c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.838698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b3b3-1114-46ea-b4b8-ce3411e553c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.839034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7b3b3-1114-46ea-b4b8-ce3411e553c6 admin admin] PATH_INFO: `/v3/projects/ee21170b05464f4a93799cc486c6f18f/users/b55ee5bbfaec4a319d957049712d0606/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.841950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 881/1786] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:29 2022] PUT /identity/v3/projects/09380167a6fd4cdea1334201b55c9537/users/d45038db2af04560b48b2156a7cc77a4/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:29.843171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404e7f3-a290-43f1-a8cb-f3aafba06a64 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:29.844097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7b3b3-1114-46ea-b4b8-ce3411e553c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee21170b05464f4a93799cc486c6f18f, user_id=b55ee5bbfaec4a319d957049712d0606, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:29.847575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7b3b3-1114-46ea-b4b8-ce3411e553c6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:29.862618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 906/1787] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:29 2022] PUT /identity/v3/projects/ee21170b05464f4a93799cc486c6f18f/users/b55ee5bbfaec4a319d957049712d0606/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:29.876803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edc17b-3d0e-413f-a8a9-b2d1970ef75b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:29.877103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edc17b-3d0e-413f-a8a9-b2d1970ef75b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:29.877452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edc17b-3d0e-413f-a8a9-b2d1970ef75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:29.921433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-85edc17b-3d0e-413f-a8a9-b2d1970ef75b None None] MFA Rules not processed for user `b55ee5bbfaec4a319d957049712d0606`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:29.922684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-85edc17b-3d0e-413f-a8a9-b2d1970ef75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:29.959342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404e7f3-a290-43f1-a8cb-f3aafba06a64 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:30.080297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:30.091315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 907/1788] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:30.105466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edc17b-3d0e-413f-a8a9-b2d1970ef75b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:30.107602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-85edc17b-3d0e-413f-a8a9-b2d1970ef75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:30.144771 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d47905-ea35-4bcd-b2ea-ababb567172c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:30.145578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d47905-ea35-4bcd-b2ea-ababb567172c admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:30.149700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d47905-ea35-4bcd-b2ea-ababb567172c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:30.150804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d47905-ea35-4bcd-b2ea-ababb567172c admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:30.152511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d47905-ea35-4bcd-b2ea-ababb567172c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:30.156735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d47905-ea35-4bcd-b2ea-ababb567172c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:30.195921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 882/1789] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:30.201446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d47905-ea35-4bcd-b2ea-ababb567172c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:30.229250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d47905-ea35-4bcd-b2ea-ababb567172c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:30.241773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f1c73-c726-4653-9869-90ef18280f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:30.243410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edc17b-3d0e-413f-a8a9-b2d1970ef75b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:30.243962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f1c73-c726-4653-9869-90ef18280f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:30.245429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f1c73-c726-4653-9869-90ef18280f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:30.246106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f1c73-c726-4653-9869-90ef18280f1c admin admin] PATH_INFO: `/v3/projects/09380167a6fd4cdea1334201b55c9537/users/d45038db2af04560b48b2156a7cc77a4/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:30.252721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f1c73-c726-4653-9869-90ef18280f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09380167a6fd4cdea1334201b55c9537, user_id=d45038db2af04560b48b2156a7cc77a4, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:30.259821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f1c73-c726-4653-9869-90ef18280f1c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:30.260480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd1fbc-a9e9-4b37-aa50-5f70e8242e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:30.261525 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd1fbc-a9e9-4b37-aa50-5f70e8242e50 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:30.261973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd1fbc-a9e9-4b37-aa50-5f70e8242e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:30.262407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd1fbc-a9e9-4b37-aa50-5f70e8242e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:30.278347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd1fbc-a9e9-4b37-aa50-5f70e8242e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:30.280623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd1fbc-a9e9-4b37-aa50-5f70e8242e50 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:30.294484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 883/1790] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:30 2022] PUT /identity/v3/projects/09380167a6fd4cdea1334201b55c9537/users/d45038db2af04560b48b2156a7cc77a4/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:30.305916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcfab9-e9c6-4ed5-968b-1b1f0125310b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:30.306424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 908/1791] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:36:30.306833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcfab9-e9c6-4ed5-968b-1b1f0125310b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:30.308074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcfab9-e9c6-4ed5-968b-1b1f0125310b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:30.405604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8edcfab9-e9c6-4ed5-968b-1b1f0125310b None None] MFA Rules not processed for user `d45038db2af04560b48b2156a7cc77a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:30.406852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8edcfab9-e9c6-4ed5-968b-1b1f0125310b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:30.440430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd1fbc-a9e9-4b37-aa50-5f70e8242e50 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:30.440430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-35bd1fbc-a9e9-4b37-aa50-5f70e8242e50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:30.459470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd1fbc-a9e9-4b37-aa50-5f70e8242e50 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:30.463384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:30.563622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:30.596114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:30.596361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:30.596531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:30.596815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] PATH_INFO: `/v3/projects/e5d6aedf27974026ba83f32cbaaba29b` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:30.608457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d6aedf27974026ba83f32cbaaba29b)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:30.612531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:30.649702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 884/1792] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:30.649702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcfab9-e9c6-4ed5-968b-1b1f0125310b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:30.664430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcfab9-e9c6-4ed5-968b-1b1f0125310b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:30.679235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:30.680286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:30.680780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:30.681296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] PATH_INFO: `/v3/users/2de6f034e5fc488fa6e10ab86b8bf271` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:30.682509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de6f034e5fc488fa6e10ab86b8bf271)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:30.693696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:30.779409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d6aedf27974026ba83f32cbaaba29b', 'request_id': 'req-7e5233d5-6658-411f-9f93-a61af8da2771'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:30.800065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:30.919995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d6aedf27974026ba83f32cbaaba29b', 'request_id': 'req-7e5233d5-6658-411f-9f93-a61af8da2771'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:30.936769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d6aedf27974026ba83f32cbaaba29b', 'request_id': 'req-7e5233d5-6658-411f-9f93-a61af8da2771'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:30.964207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] The token cache is being invalidate because project e5d6aedf27974026ba83f32cbaaba29b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:30.964403 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:30.976633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:30.984790 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 909/1793] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:30 2022] DELETE /identity/v3/projects/e5d6aedf27974026ba83f32cbaaba29b => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:30.988693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:30.995014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:31.012547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:31.025541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de6f034e5fc488fa6e10ab86b8bf271'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:31.027800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de6f034e5fc488fa6e10ab86b8bf271'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:31.073990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de6f034e5fc488fa6e10ab86b8bf271'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:31.075121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de6f034e5fc488fa6e10ab86b8bf271'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:31.114135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25b08508-e0cb-40ed-bfe8-36bf99f624ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de6f034e5fc488fa6e10ab86b8bf271'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:31.122001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5233d5-6658-411f-9f93-a61af8da2771 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:31.138623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a0006-b3f9-4f71-83d6-3e1250d41052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:31.145514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a0006-b3f9-4f71-83d6-3e1250d41052 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:31.145514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a0006-b3f9-4f71-83d6-3e1250d41052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:31.146749 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a0006-b3f9-4f71-83d6-3e1250d41052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:31.149685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-aa5a0006-b3f9-4f71-83d6-3e1250d41052 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:31.193894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:31.224840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 885/1794] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:30 2022] DELETE /identity/v3/users/2de6f034e5fc488fa6e10ab86b8bf271 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:31.226875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062b250-a2b1-4381-ad53-3486cc6aed6a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:31.227317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062b250-a2b1-4381-ad53-3486cc6aed6a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:31.233486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062b250-a2b1-4381-ad53-3486cc6aed6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:31.315014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-6062b250-a2b1-4381-ad53-3486cc6aed6a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:31.320395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6062b250-a2b1-4381-ad53-3486cc6aed6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:31.368259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a0006-b3f9-4f71-83d6-3e1250d41052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:31.370514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a0006-b3f9-4f71-83d6-3e1250d41052 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:31.387553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 910/1795] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:30 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:31.389667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890fbf1-1f73-42dc-8bc2-5b6e566122a3 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:31.396481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890fbf1-1f73-42dc-8bc2-5b6e566122a3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:31.396481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890fbf1-1f73-42dc-8bc2-5b6e566122a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:31.463817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-1890fbf1-1f73-42dc-8bc2-5b6e566122a3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:31.472903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1890fbf1-1f73-42dc-8bc2-5b6e566122a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:31.522114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 886/1796] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:31.532684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e09a8e-b7f5-45ab-b739-704990506aac None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:31.532934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e09a8e-b7f5-45ab-b739-704990506aac None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:31.533221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e09a8e-b7f5-45ab-b739-704990506aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:31.557997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-59e09a8e-b7f5-45ab-b739-704990506aac None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:31.559254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-59e09a8e-b7f5-45ab-b739-704990506aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:31.570383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 887/1797] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:31.575755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e09a8e-b7f5-45ab-b739-704990506aac None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:31.589235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e09a8e-b7f5-45ab-b739-704990506aac None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:31.594683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:31.595817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:31.596119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:31.596477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:31.597245 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:31.597741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:31.599686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:31.628091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 888/1798] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:31.628091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:31.630245 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:31.652107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 911/1799] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:31.658043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890fbf1-1f73-42dc-8bc2-5b6e566122a3 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:31.697185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890fbf1-1f73-42dc-8bc2-5b6e566122a3 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:31.707093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af5873-71bd-400a-b6cb-d638f2a45e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:31.709701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5873-71bd-400a-b6cb-d638f2a45e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:31.709701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5873-71bd-400a-b6cb-d638f2a45e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:31.709701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5873-71bd-400a-b6cb-d638f2a45e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:31.709701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af5873-71bd-400a-b6cb-d638f2a45e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:31.711641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af5873-71bd-400a-b6cb-d638f2a45e6f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:31.712950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3ceb5-f4e6-427e-ba55-0d34e346bec0 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:31.718340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965f7c8-e347-4f56-9a06-0fac397a0692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:31.719070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f7c8-e347-4f56-9a06-0fac397a0692 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:31.719649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f7c8-e347-4f56-9a06-0fac397a0692 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:31.719929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f7c8-e347-4f56-9a06-0fac397a0692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:31.722881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d965f7c8-e347-4f56-9a06-0fac397a0692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:31.855893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965f7c8-e347-4f56-9a06-0fac397a0692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:31.858399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965f7c8-e347-4f56-9a06-0fac397a0692 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:31.877813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 889/1800] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:31.880666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fa131-d41a-4e9b-ae23-6bb978301cb9 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:31.885401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fa131-d41a-4e9b-ae23-6bb978301cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:31.885401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fa131-d41a-4e9b-ae23-6bb978301cb9 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:31.885401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 890/1801] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:36:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:36:31.885401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fa131-d41a-4e9b-ae23-6bb978301cb9 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:31.888845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b23fa131-d41a-4e9b-ae23-6bb978301cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:31.890113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 912/1802] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:31 2022] POST /identity/v3/projects => generated 361 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:31.895982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af5873-71bd-400a-b6cb-d638f2a45e6f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:31.982586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fa131-d41a-4e9b-ae23-6bb978301cb9 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:31.995146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a19e9-3acb-45b6-96ed-d5db80c92686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:31.996002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a19e9-3acb-45b6-96ed-d5db80c92686 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:31.997337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a19e9-3acb-45b6-96ed-d5db80c92686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:32.001976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a19e9-3acb-45b6-96ed-d5db80c92686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:32.003423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:32.052480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:32.082209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af5873-71bd-400a-b6cb-d638f2a45e6f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:32.098559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2b2ce-79f3-4f60-bb01-c8e53a9ee423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:32.104848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2b2ce-79f3-4f60-bb01-c8e53a9ee423 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:32.104848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2b2ce-79f3-4f60-bb01-c8e53a9ee423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:32.104848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2b2ce-79f3-4f60-bb01-c8e53a9ee423 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:32.104848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2b2ce-79f3-4f60-bb01-c8e53a9ee423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:32.105173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2b2ce-79f3-4f60-bb01-c8e53a9ee423 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:32.158008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a19e9-3acb-45b6-96ed-d5db80c92686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:32.160048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a19e9-3acb-45b6-96ed-d5db80c92686 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:32.184909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 891/1803] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:32.186988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c45a0a-fa99-4729-8b20-6565e50c7a9f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:32.188585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c45a0a-fa99-4729-8b20-6565e50c7a9f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:32.189069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c45a0a-fa99-4729-8b20-6565e50c7a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:32.222653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 913/1804] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:31 2022] POST /identity/v3/users => generated 281 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:32.227364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-18c45a0a-fa99-4729-8b20-6565e50c7a9f None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:32.228643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2b2ce-79f3-4f60-bb01-c8e53a9ee423 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:32.229134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-18c45a0a-fa99-4729-8b20-6565e50c7a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:32.252673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2b2ce-79f3-4f60-bb01-c8e53a9ee423 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:32.276068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26af1b5-3b2a-47ab-912e-742ce6e1d99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:32.276797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26af1b5-3b2a-47ab-912e-742ce6e1d99e admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:32.277316 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26af1b5-3b2a-47ab-912e-742ce6e1d99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:32.277471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26af1b5-3b2a-47ab-912e-742ce6e1d99e admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:32.279251 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26af1b5-3b2a-47ab-912e-742ce6e1d99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:32.283342 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26af1b5-3b2a-47ab-912e-742ce6e1d99e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:32.327898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 914/1805] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:32.328953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26af1b5-3b2a-47ab-912e-742ce6e1d99e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:32.350557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26af1b5-3b2a-47ab-912e-742ce6e1d99e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:32.361746 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f269af4-cbd0-4d62-9dc3-d23f86673bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:32.363948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f269af4-cbd0-4d62-9dc3-d23f86673bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:32.363948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f269af4-cbd0-4d62-9dc3-d23f86673bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:32.363948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f269af4-cbd0-4d62-9dc3-d23f86673bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:32.363948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f269af4-cbd0-4d62-9dc3-d23f86673bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:32.369458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f269af4-cbd0-4d62-9dc3-d23f86673bfc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:32.394905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 915/1806] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:32.399920 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f269af4-cbd0-4d62-9dc3-d23f86673bfc admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:32.423748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 892/1807] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:36:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:32.426292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f269af4-cbd0-4d62-9dc3-d23f86673bfc admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:32.447973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceee30-9cec-46cd-bc01-2638b97a4e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:32.451114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceee30-9cec-46cd-bc01-2638b97a4e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:32.456113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceee30-9cec-46cd-bc01-2638b97a4e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:32.456525 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceee30-9cec-46cd-bc01-2638b97a4e6e admin admin] PATH_INFO: `/v3/projects/e0569d9a7b2546d38afa4f54536b6149/users/3873a0d443c144bbb224bf35895bcc72/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:32.512317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceee30-9cec-46cd-bc01-2638b97a4e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0569d9a7b2546d38afa4f54536b6149, user_id=3873a0d443c144bbb224bf35895bcc72, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:32.516737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceee30-9cec-46cd-bc01-2638b97a4e6e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:32.607458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 916/1808] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:32 2022] PUT /identity/v3/projects/e0569d9a7b2546d38afa4f54536b6149/users/3873a0d443c144bbb224bf35895bcc72/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:32.609506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f768cd7-3f9d-4356-baa6-72010ff30486 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:32.610377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f768cd7-3f9d-4356-baa6-72010ff30486 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:32.610377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f768cd7-3f9d-4356-baa6-72010ff30486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:32.697559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8f768cd7-3f9d-4356-baa6-72010ff30486 None None] MFA Rules not processed for user `3873a0d443c144bbb224bf35895bcc72`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:32.713653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8f768cd7-3f9d-4356-baa6-72010ff30486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:32.904600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 893/1809] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:32.946080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceee30-9cec-46cd-bc01-2638b97a4e6e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:33.036015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceee30-9cec-46cd-bc01-2638b97a4e6e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:33.041734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d7c95-687d-47d3-9294-8afdc086af02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:33.042481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d7c95-687d-47d3-9294-8afdc086af02 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:33.042872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d7c95-687d-47d3-9294-8afdc086af02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:33.043293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d7c95-687d-47d3-9294-8afdc086af02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:33.063488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d7c95-687d-47d3-9294-8afdc086af02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:33.067320 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d7c95-687d-47d3-9294-8afdc086af02 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:33.097951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 917/1810] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:33.121684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f768cd7-3f9d-4356-baa6-72010ff30486 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:33.144559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:33.145690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d7c95-687d-47d3-9294-8afdc086af02 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:33.151421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9c7d7c95-687d-47d3-9294-8afdc086af02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:33.258301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f768cd7-3f9d-4356-baa6-72010ff30486 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:33.263283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:33.264202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:33.264486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:33.264765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] PATH_INFO: `/v3/users/3873a0d443c144bbb224bf35895bcc72` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:33.266255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3873a0d443c144bbb224bf35895bcc72)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:33.268345 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:33.470850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d7c95-687d-47d3-9294-8afdc086af02 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:33.515838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24edd7d-c74f-4a6d-84ba-2c42ec50090e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:33.517037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24edd7d-c74f-4a6d-84ba-2c42ec50090e service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:33.517723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24edd7d-c74f-4a6d-84ba-2c42ec50090e service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:33.519624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24edd7d-c74f-4a6d-84ba-2c42ec50090e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:33.536164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24edd7d-c74f-4a6d-84ba-2c42ec50090e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:33.537824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3873a0d443c144bbb224bf35895bcc72'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:33.538248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24edd7d-c74f-4a6d-84ba-2c42ec50090e service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:33.538565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3873a0d443c144bbb224bf35895bcc72'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:33.557712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 918/1811] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:36:33.566407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3873a0d443c144bbb224bf35895bcc72'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:33.566407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3873a0d443c144bbb224bf35895bcc72'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:33.570728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24edd7d-c74f-4a6d-84ba-2c42ec50090e service nova] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:33.580634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c24edd7d-c74f-4a6d-84ba-2c42ec50090e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:33.612344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-25ab9145-3b22-4e96-92ab-c8def92c3720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3873a0d443c144bbb224bf35895bcc72'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:33.625984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:33.648028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24edd7d-c74f-4a6d-84ba-2c42ec50090e service nova] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:33.671489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 894/1812] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:33 2022] DELETE /identity/v3/users/3873a0d443c144bbb224bf35895bcc72 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:33.677077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73902-d7f6-48ab-8c60-587e9a113a44 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:33.680397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73902-d7f6-48ab-8c60-587e9a113a44 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:33.681371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73902-d7f6-48ab-8c60-587e9a113a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:33.684696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:33.685901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:33.696060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:33.697766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] PATH_INFO: `/v3/projects/81f0d69f219148e4a8b49c2465e25d08` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:33.704022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=81f0d69f219148e4a8b49c2465e25d08)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:33.710757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:33.738780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-3ea73902-d7f6-48ab-8c60-587e9a113a44 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:33.744146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3ea73902-d7f6-48ab-8c60-587e9a113a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:33.916654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81f0d69f219148e4a8b49c2465e25d08', 'request_id': 'req-da40be7c-a66f-4132-9b23-38c21d2b9a4a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:33.926051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 895/1813] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:33.930585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7b1a2-023d-4ca8-bfb6-6c2edecb7c34 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:33.930585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7b1a2-023d-4ca8-bfb6-6c2edecb7c34 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:33.930585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7b1a2-023d-4ca8-bfb6-6c2edecb7c34 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:33.931476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 896/1814] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:36:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:36:33.969600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782d0eb-bc2e-4fec-96b1-dd168a97af80 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:33.969600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782d0eb-bc2e-4fec-96b1-dd168a97af80 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:33.969600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782d0eb-bc2e-4fec-96b1-dd168a97af80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:33.999426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81f0d69f219148e4a8b49c2465e25d08', 'request_id': 'req-da40be7c-a66f-4132-9b23-38c21d2b9a4a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:33.999706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e782d0eb-bc2e-4fec-96b1-dd168a97af80 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:34.001072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e782d0eb-bc2e-4fec-96b1-dd168a97af80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:34.006615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81f0d69f219148e4a8b49c2465e25d08', 'request_id': 'req-da40be7c-a66f-4132-9b23-38c21d2b9a4a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:34.040264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] The token cache is being invalidate because project 81f0d69f219148e4a8b49c2465e25d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:34.040558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:34.041343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:34.043771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 919/1815] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:33 2022] DELETE /identity/v3/projects/81f0d69f219148e4a8b49c2465e25d08 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:34.044772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:34.046367 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:34.170983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 897/1816] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:36:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:34.182250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa640a-3dc8-4449-a45a-7cd047c0d5a9 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:34.182724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa640a-3dc8-4449-a45a-7cd047c0d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:34.182922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa640a-3dc8-4449-a45a-7cd047c0d5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:34.222016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-65aa640a-3dc8-4449-a45a-7cd047c0d5a9 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:34.227247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-65aa640a-3dc8-4449-a45a-7cd047c0d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:34.236030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40be7c-a66f-4132-9b23-38c21d2b9a4a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:34.264300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70de0f7-b6d9-4e93-8f12-0032db796317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:34.265133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70de0f7-b6d9-4e93-8f12-0032db796317 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:34.265424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70de0f7-b6d9-4e93-8f12-0032db796317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:34.265720 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70de0f7-b6d9-4e93-8f12-0032db796317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:34.268605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e70de0f7-b6d9-4e93-8f12-0032db796317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:34.344511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 898/1817] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:34.349387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa640a-3dc8-4449-a45a-7cd047c0d5a9 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:34.354305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-65aa640a-3dc8-4449-a45a-7cd047c0d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:34.387991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa640a-3dc8-4449-a45a-7cd047c0d5a9 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:34.403101 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b948c4b-7f74-48b3-abae-25184e06d66a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:34.403796 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b948c4b-7f74-48b3-abae-25184e06d66a service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:34.404077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b948c4b-7f74-48b3-abae-25184e06d66a service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:34.404348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b948c4b-7f74-48b3-abae-25184e06d66a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:34.416374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70de0f7-b6d9-4e93-8f12-0032db796317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:34.416735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70de0f7-b6d9-4e93-8f12-0032db796317 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:34.424128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b948c4b-7f74-48b3-abae-25184e06d66a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:34.426464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b948c4b-7f74-48b3-abae-25184e06d66a service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:34.427700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 920/1818] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:34.429308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5b4c9-e378-47e7-9490-7ea92f12ad89 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:34.429606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5b4c9-e378-47e7-9490-7ea92f12ad89 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:34.429887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5b4c9-e378-47e7-9490-7ea92f12ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:34.443827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 899/1819] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:34.491783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5ab5b4c9-e378-47e7-9490-7ea92f12ad89 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:34.496649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5ab5b4c9-e378-47e7-9490-7ea92f12ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:34.525879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 921/1820] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:34.531855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b948c4b-7f74-48b3-abae-25184e06d66a service nova] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:34.560202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b948c4b-7f74-48b3-abae-25184e06d66a service nova] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:34.566256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:34.566995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:34.567345 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:34.567782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:34.568713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:34.569625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:34.571858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:34.585734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 900/1821] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:34.590025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5b4c9-e378-47e7-9490-7ea92f12ad89 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:34.599454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5b4c9-e378-47e7-9490-7ea92f12ad89 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:34.610143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f72c9-4131-4dc8-b98d-e19bc3f4623a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:34.610143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f72c9-4131-4dc8-b98d-e19bc3f4623a admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:34.610143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f72c9-4131-4dc8-b98d-e19bc3f4623a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:34.610143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f72c9-4131-4dc8-b98d-e19bc3f4623a admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:34.611749 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f72c9-4131-4dc8-b98d-e19bc3f4623a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:34.612867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:34.616875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:34.621092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f72c9-4131-4dc8-b98d-e19bc3f4623a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:34.650489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b06709-f599-47a5-b483-9c667b2c9a3b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:34.660025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:34.661070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:34.661358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:34.661640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] PATH_INFO: `/v3/projects/799a3b48da7542b1bc77727a36f5568c` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:34.662979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=799a3b48da7542b1bc77727a36f5568c)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:34.665187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:34.789281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 922/1822] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:34 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:34.792217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f72c9-4131-4dc8-b98d-e19bc3f4623a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:34.824933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '799a3b48da7542b1bc77727a36f5568c', 'request_id': 'req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:34.831896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '799a3b48da7542b1bc77727a36f5568c', 'request_id': 'req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:34.840134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '799a3b48da7542b1bc77727a36f5568c', 'request_id': 'req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:34.861218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f72c9-4131-4dc8-b98d-e19bc3f4623a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:34.915472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:34.919647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05f1b5-0d47-471d-8179-45f66fef6e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:34.919852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] The token cache is being invalidate because project 799a3b48da7542b1bc77727a36f5568c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:34.927288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:34.927407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05f1b5-0d47-471d-8179-45f66fef6e9e admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:34.927407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05f1b5-0d47-471d-8179-45f66fef6e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:34.927579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:34.927654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05f1b5-0d47-471d-8179-45f66fef6e9e admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:34.927654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05f1b5-0d47-471d-8179-45f66fef6e9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:34.927795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 901/1823] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:34 2022] DELETE /identity/v3/projects/799a3b48da7542b1bc77727a36f5568c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:34.927862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05f1b5-0d47-471d-8179-45f66fef6e9e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:34.927926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:34.927926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:34.940652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783a6b0-2dbd-46ad-bcc5-f0ba6512a914 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:34.946173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:34.947030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:34.947311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:34.947613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] PATH_INFO: `/v3/users/b55ee5bbfaec4a319d957049712d0606` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:34.948812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b55ee5bbfaec4a319d957049712d0606)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:34.950814 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.012927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 923/1824] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:34 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:35.014184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05f1b5-0d47-471d-8179-45f66fef6e9e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.017517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-be05f1b5-0d47-471d-8179-45f66fef6e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:35.031581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05f1b5-0d47-471d-8179-45f66fef6e9e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:35.036323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:35.037174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.037613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.037791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] PATH_INFO: `/v3/projects/e0569d9a7b2546d38afa4f54536b6149` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.039301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0569d9a7b2546d38afa4f54536b6149)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:35.041268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.188362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:35.205194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b55ee5bbfaec4a319d957049712d0606'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.205904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b55ee5bbfaec4a319d957049712d0606'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.210698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0569d9a7b2546d38afa4f54536b6149', 'request_id': 'req-9547b07c-38c5-46ad-8da8-d7462d193c0c'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.214709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b55ee5bbfaec4a319d957049712d0606'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.215470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b55ee5bbfaec4a319d957049712d0606'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.228214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-05c985e3-44f7-412c-a49c-c94117bd58c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b55ee5bbfaec4a319d957049712d0606'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.275112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0569d9a7b2546d38afa4f54536b6149', 'request_id': 'req-9547b07c-38c5-46ad-8da8-d7462d193c0c'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.281129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0569d9a7b2546d38afa4f54536b6149', 'request_id': 'req-9547b07c-38c5-46ad-8da8-d7462d193c0c'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.293847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] The token cache is being invalidate because project e0569d9a7b2546d38afa4f54536b6149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:35.297859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.297859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:35.297859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 924/1825] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:35 2022] DELETE /identity/v3/projects/e0569d9a7b2546d38afa4f54536b6149 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:35.297859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.298441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:35.343797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 902/1826] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:34 2022] DELETE /identity/v3/users/b55ee5bbfaec4a319d957049712d0606 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:35.345219 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa19f65-b213-440a-888e-ccb2a815cccc None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.346727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa19f65-b213-440a-888e-ccb2a815cccc None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.347014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa19f65-b213-440a-888e-ccb2a815cccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.372681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-efa19f65-b213-440a-888e-ccb2a815cccc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:35.376032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-efa19f65-b213-440a-888e-ccb2a815cccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:35.379703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547b07c-38c5-46ad-8da8-d7462d193c0c admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:35.445929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0414f-7769-41da-a0ca-311d925ab157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:35.446743 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0414f-7769-41da-a0ca-311d925ab157 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.447150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0414f-7769-41da-a0ca-311d925ab157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.447436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0414f-7769-41da-a0ca-311d925ab157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.448205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0414f-7769-41da-a0ca-311d925ab157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:35.455061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 903/1827] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:35.455214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0414f-7769-41da-a0ca-311d925ab157 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.457040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea1e3a-e1fb-4a29-89fc-e918eaa8860f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.457330 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea1e3a-e1fb-4a29-89fc-e918eaa8860f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.457619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea1e3a-e1fb-4a29-89fc-e918eaa8860f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.464167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 925/1828] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:35.465373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0414f-7769-41da-a0ca-311d925ab157 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.477495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0414f-7769-41da-a0ca-311d925ab157 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:35.480387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-bcea1e3a-e1fb-4a29-89fc-e918eaa8860f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:35.480549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-bcea1e3a-e1fb-4a29-89fc-e918eaa8860f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:35.482673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c99d98-8eb4-4c8c-b7d3-3a265b2b90e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:35.483417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c99d98-8eb4-4c8c-b7d3-3a265b2b90e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.483743 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c99d98-8eb4-4c8c-b7d3-3a265b2b90e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.484022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c99d98-8eb4-4c8c-b7d3-3a265b2b90e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.484854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c99d98-8eb4-4c8c-b7d3-3a265b2b90e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:35.486545 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c99d98-8eb4-4c8c-b7d3-3a265b2b90e9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.488817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 904/1829] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:35.492998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea1e3a-e1fb-4a29-89fc-e918eaa8860f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.501076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 926/1830] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:35.502187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea1e3a-e1fb-4a29-89fc-e918eaa8860f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:35.504206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c99d98-8eb4-4c8c-b7d3-3a265b2b90e9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.507903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac2121-7cf8-4e87-b823-aa81ec5b9ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:35.508569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac2121-7cf8-4e87-b823-aa81ec5b9ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.508847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac2121-7cf8-4e87-b823-aa81ec5b9ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.509160 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac2121-7cf8-4e87-b823-aa81ec5b9ff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.509978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac2121-7cf8-4e87-b823-aa81ec5b9ff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:35.510375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac2121-7cf8-4e87-b823-aa81ec5b9ff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:35.511983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac2121-7cf8-4e87-b823-aa81ec5b9ff4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.517296 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c99d98-8eb4-4c8c-b7d3-3a265b2b90e9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:35.523164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7855c51-e9a0-415e-aa67-aa0775da7fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:35.524099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7855c51-e9a0-415e-aa67-aa0775da7fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.524381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7855c51-e9a0-415e-aa67-aa0775da7fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.524658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7855c51-e9a0-415e-aa67-aa0775da7fd2 admin admin] PATH_INFO: `/v3/projects/c00a0cb2e37c4689ace91bc2a1086166/users/9bb2774a5cbd40598462d019fd330c3a/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.525678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 905/1831] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:35.531900 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac2121-7cf8-4e87-b823-aa81ec5b9ff4 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.541140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7855c51-e9a0-415e-aa67-aa0775da7fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00a0cb2e37c4689ace91bc2a1086166, user_id=9bb2774a5cbd40598462d019fd330c3a, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:35.543945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7855c51-e9a0-415e-aa67-aa0775da7fd2 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.544490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac2121-7cf8-4e87-b823-aa81ec5b9ff4 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:35.552274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32b2fb-6f9c-4dd8-94ff-c5467035ce11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:35.553634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32b2fb-6f9c-4dd8-94ff-c5467035ce11 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.553634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32b2fb-6f9c-4dd8-94ff-c5467035ce11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.554059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32b2fb-6f9c-4dd8-94ff-c5467035ce11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.554840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32b2fb-6f9c-4dd8-94ff-c5467035ce11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:35.557358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32b2fb-6f9c-4dd8-94ff-c5467035ce11 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.625573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 927/1832] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:35 2022] PUT /identity/v3/projects/c00a0cb2e37c4689ace91bc2a1086166/users/9bb2774a5cbd40598462d019fd330c3a/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:35.641070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedea72-dcdf-4e63-8627-c4340b4150ab None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.641070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedea72-dcdf-4e63-8627-c4340b4150ab None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.643377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedea72-dcdf-4e63-8627-c4340b4150ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.671817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3bedea72-dcdf-4e63-8627-c4340b4150ab None None] MFA Rules not processed for user `9bb2774a5cbd40598462d019fd330c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:35.673065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3bedea72-dcdf-4e63-8627-c4340b4150ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:35.699207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 906/1833] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:35 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:35.705142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32b2fb-6f9c-4dd8-94ff-c5467035ce11 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.750445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 928/1834] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:35.755085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedea72-dcdf-4e63-8627-c4340b4150ab None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.790301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32b2fb-6f9c-4dd8-94ff-c5467035ce11 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:35.795349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670a1b2-139f-4329-9464-f1e2a43be18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:35.796184 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670a1b2-139f-4329-9464-f1e2a43be18d admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.796466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670a1b2-139f-4329-9464-f1e2a43be18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.796739 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670a1b2-139f-4329-9464-f1e2a43be18d admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.797450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670a1b2-139f-4329-9464-f1e2a43be18d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:35.799377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670a1b2-139f-4329-9464-f1e2a43be18d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.839321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedea72-dcdf-4e63-8627-c4340b4150ab None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:35.845865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4314ad-5451-4922-ae2b-43432c684c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:35.845865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4314ad-5451-4922-ae2b-43432c684c20 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.845865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4314ad-5451-4922-ae2b-43432c684c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.845865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4314ad-5451-4922-ae2b-43432c684c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.846173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4314ad-5451-4922-ae2b-43432c684c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:35.846173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4314ad-5451-4922-ae2b-43432c684c20 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.880326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 907/1835] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:35 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:35.881501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670a1b2-139f-4329-9464-f1e2a43be18d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.884185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8670a1b2-139f-4329-9464-f1e2a43be18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:35.899391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670a1b2-139f-4329-9464-f1e2a43be18d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:35.904783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:35.905657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:35.905969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:35.906273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] PATH_INFO: `/v3/projects/ee21170b05464f4a93799cc486c6f18f` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:35.907496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee21170b05464f4a93799cc486c6f18f)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:35.909806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:35.928403 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 929/1836] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:35 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:35.932698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4314ad-5451-4922-ae2b-43432c684c20 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:35.995484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4314ad-5451-4922-ae2b-43432c684c20 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.000327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de629804-ea6c-4982-bebb-9b954a9c202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.000993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de629804-ea6c-4982-bebb-9b954a9c202d admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.001280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de629804-ea6c-4982-bebb-9b954a9c202d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.001562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de629804-ea6c-4982-bebb-9b954a9c202d admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.002283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de629804-ea6c-4982-bebb-9b954a9c202d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.003989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de629804-ea6c-4982-bebb-9b954a9c202d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.014397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 930/1837] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:36.015210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de629804-ea6c-4982-bebb-9b954a9c202d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.017903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee21170b05464f4a93799cc486c6f18f', 'request_id': 'req-b71fff05-a5c4-437b-b987-586244ad69cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:36.024319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee21170b05464f4a93799cc486c6f18f', 'request_id': 'req-b71fff05-a5c4-437b-b987-586244ad69cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:36.025341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de629804-ea6c-4982-bebb-9b954a9c202d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.031144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee21170b05464f4a93799cc486c6f18f', 'request_id': 'req-b71fff05-a5c4-437b-b987-586244ad69cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:36.079943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ead42-a759-4b03-b80e-24d5bca93fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.080606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ead42-a759-4b03-b80e-24d5bca93fed admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.080888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ead42-a759-4b03-b80e-24d5bca93fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.081163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ead42-a759-4b03-b80e-24d5bca93fed admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.081845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ead42-a759-4b03-b80e-24d5bca93fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.083815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ead42-a759-4b03-b80e-24d5bca93fed admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.095439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] The token cache is being invalidate because project ee21170b05464f4a93799cc486c6f18f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:36.095755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:36.096415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:36.097996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 908/1838] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:35 2022] DELETE /identity/v3/projects/ee21170b05464f4a93799cc486c6f18f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:36.098762 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.100066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:36.114457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fff05-a5c4-437b-b987-586244ad69cf admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.119479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e95b4d-8722-4a28-be61-20e92eb7b90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.120249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e95b4d-8722-4a28-be61-20e92eb7b90c admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.120540 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e95b4d-8722-4a28-be61-20e92eb7b90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.120821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e95b4d-8722-4a28-be61-20e92eb7b90c admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.121554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e95b4d-8722-4a28-be61-20e92eb7b90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.123143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e95b4d-8722-4a28-be61-20e92eb7b90c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.137626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 909/1839] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:36.137626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ed028-4d29-43b6-b7d0-5af1449213e9 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.137626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ed028-4d29-43b6-b7d0-5af1449213e9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.137626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ed028-4d29-43b6-b7d0-5af1449213e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.156189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-2f1ed028-4d29-43b6-b7d0-5af1449213e9 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:36.157409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2f1ed028-4d29-43b6-b7d0-5af1449213e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:36.173936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 931/1840] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:36 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:36.174196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 910/1841] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:36.174270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ead42-a759-4b03-b80e-24d5bca93fed admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.175493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ed028-4d29-43b6-b7d0-5af1449213e9 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.178943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:36.180729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2f1ed028-4d29-43b6-b7d0-5af1449213e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:36.186148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ead42-a759-4b03-b80e-24d5bca93fed admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.191788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dcbc0-4413-4ae2-9184-b48a66854b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.192740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dcbc0-4413-4ae2-9184-b48a66854b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.192971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ed028-4d29-43b6-b7d0-5af1449213e9 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.193241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dcbc0-4413-4ae2-9184-b48a66854b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.193607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dcbc0-4413-4ae2-9184-b48a66854b9c admin admin] PATH_INFO: `/v3/projects/b41ab9ca798f42d0be421d7e02ab694e/users/d4aaefcce9914009af255400c9557875/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.198424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df08886-43aa-4c8e-a4bc-e8274e06edeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.199099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df08886-43aa-4c8e-a4bc-e8274e06edeb admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.199382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df08886-43aa-4c8e-a4bc-e8274e06edeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.199706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df08886-43aa-4c8e-a4bc-e8274e06edeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.200440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df08886-43aa-4c8e-a4bc-e8274e06edeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.202048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df08886-43aa-4c8e-a4bc-e8274e06edeb admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.211594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dcbc0-4413-4ae2-9184-b48a66854b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41ab9ca798f42d0be421d7e02ab694e, user_id=d4aaefcce9914009af255400c9557875, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.214257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dcbc0-4413-4ae2-9184-b48a66854b9c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.215553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 911/1842] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:36.216956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec283-fe08-4229-9549-3b2f9b07154a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.217322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec283-fe08-4229-9549-3b2f9b07154a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.217617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec283-fe08-4229-9549-3b2f9b07154a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.235865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-924ec283-fe08-4229-9549-3b2f9b07154a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:36.237063 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-924ec283-fe08-4229-9549-3b2f9b07154a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:36.245282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 912/1843] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:36.249200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec283-fe08-4229-9549-3b2f9b07154a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.255607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec283-fe08-4229-9549-3b2f9b07154a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.260733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad9501-a39d-4656-bab3-5c76553928d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.261581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad9501-a39d-4656-bab3-5c76553928d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.261859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad9501-a39d-4656-bab3-5c76553928d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.262129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad9501-a39d-4656-bab3-5c76553928d7 admin admin] PATH_INFO: `/v3/projects/f307f719ef0e454f9b4c40ea9a57dae4/users/90c3fbee94f14235940660245c144518/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.274230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 932/1844] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:36 2022] PUT /identity/v3/projects/b41ab9ca798f42d0be421d7e02ab694e/users/d4aaefcce9914009af255400c9557875/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:36.279058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dcbc0-4413-4ae2-9184-b48a66854b9c admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.279715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad9501-a39d-4656-bab3-5c76553928d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f307f719ef0e454f9b4c40ea9a57dae4, user_id=90c3fbee94f14235940660245c144518, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.281281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad9501-a39d-4656-bab3-5c76553928d7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.344021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dcbc0-4413-4ae2-9184-b48a66854b9c admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.348913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8b3a7-5216-4af0-9158-306da94f3a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.349576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8b3a7-5216-4af0-9158-306da94f3a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.349856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8b3a7-5216-4af0-9158-306da94f3a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.350167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8b3a7-5216-4af0-9158-306da94f3a02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.350952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8b3a7-5216-4af0-9158-306da94f3a02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:36.351260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8b3a7-5216-4af0-9158-306da94f3a02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.352887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8b3a7-5216-4af0-9158-306da94f3a02 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.364802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 913/1845] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:36 2022] PUT /identity/v3/projects/f307f719ef0e454f9b4c40ea9a57dae4/users/90c3fbee94f14235940660245c144518/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:36.365561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 933/1846] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:36.369174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8b3a7-5216-4af0-9158-306da94f3a02 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.369302 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0081b-573d-4a41-8aca-3684cf584c47 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.369518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0081b-573d-4a41-8aca-3684cf584c47 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.369806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0081b-573d-4a41-8aca-3684cf584c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.378899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8b3a7-5216-4af0-9158-306da94f3a02 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.396416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-cfe0081b-573d-4a41-8aca-3684cf584c47 None None] MFA Rules not processed for user `d4aaefcce9914009af255400c9557875`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:36.397745 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-cfe0081b-573d-4a41-8aca-3684cf584c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:36.440131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865ada6-d68a-433e-a64d-006cf2e9ba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.440847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865ada6-d68a-433e-a64d-006cf2e9ba71 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.445160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865ada6-d68a-433e-a64d-006cf2e9ba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.445160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865ada6-d68a-433e-a64d-006cf2e9ba71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.445160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865ada6-d68a-433e-a64d-006cf2e9ba71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.445160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865ada6-d68a-433e-a64d-006cf2e9ba71 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.456372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 934/1847] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:36.457410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865ada6-d68a-433e-a64d-006cf2e9ba71 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.475167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865ada6-d68a-433e-a64d-006cf2e9ba71 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.475634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 914/1848] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:36.475634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0081b-573d-4a41-8aca-3684cf584c47 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.487608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0081b-573d-4a41-8aca-3684cf584c47 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.491503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:36.493585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadfc54-c001-45c6-9051-277502e3fe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.494181 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098125a7-085d-4676-9f36-9ecf463807ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.494625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadfc54-c001-45c6-9051-277502e3fe25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.495161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadfc54-c001-45c6-9051-277502e3fe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.495279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098125a7-085d-4676-9f36-9ecf463807ee admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.495364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadfc54-c001-45c6-9051-277502e3fe25 admin admin] PATH_INFO: `/v3/projects/f307f719ef0e454f9b4c40ea9a57dae4/users/90c3fbee94f14235940660245c144518/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.495436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098125a7-085d-4676-9f36-9ecf463807ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.501091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098125a7-085d-4676-9f36-9ecf463807ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.501091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098125a7-085d-4676-9f36-9ecf463807ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.501091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098125a7-085d-4676-9f36-9ecf463807ee admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.501915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadfc54-c001-45c6-9051-277502e3fe25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f307f719ef0e454f9b4c40ea9a57dae4, user_id=90c3fbee94f14235940660245c144518, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.503768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadfc54-c001-45c6-9051-277502e3fe25 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.519663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 915/1849] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:36 2022] PUT /identity/v3/projects/f307f719ef0e454f9b4c40ea9a57dae4/users/90c3fbee94f14235940660245c144518/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:36.519663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadfc54-c001-45c6-9051-277502e3fe25 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.611656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 935/1850] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:36 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:36.613311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d3759-f2ff-402c-a7b9-6b9160dd16b8 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.613606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d3759-f2ff-402c-a7b9-6b9160dd16b8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.613886 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d3759-f2ff-402c-a7b9-6b9160dd16b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.651828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadfc54-c001-45c6-9051-277502e3fe25 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.652539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4c0d3759-f2ff-402c-a7b9-6b9160dd16b8 None None] MFA Rules not processed for user `90c3fbee94f14235940660245c144518`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:36.653824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4c0d3759-f2ff-402c-a7b9-6b9160dd16b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:36.730060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d0ba6-5cdd-4dd3-8458-015872db9026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.731260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d0ba6-5cdd-4dd3-8458-015872db9026 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:36.731814 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d0ba6-5cdd-4dd3-8458-015872db9026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:36.732403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d0ba6-5cdd-4dd3-8458-015872db9026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:36.733454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d0ba6-5cdd-4dd3-8458-015872db9026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:36.736221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d0ba6-5cdd-4dd3-8458-015872db9026 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:36.835892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 916/1851] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:36 2022] POST /identity/v3/projects => generated 373 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:36.837082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d0ba6-5cdd-4dd3-8458-015872db9026 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.910170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 936/1852] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:36.911747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d3759-f2ff-402c-a7b9-6b9160dd16b8 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:36.926002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d3759-f2ff-402c-a7b9-6b9160dd16b8 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.991969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d0ba6-5cdd-4dd3-8458-015872db9026 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:36.999231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdd760-a136-401e-8e16-4ade2bb103fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:36.999996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdd760-a136-401e-8e16-4ade2bb103fd admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.000281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdd760-a136-401e-8e16-4ade2bb103fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.000557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdd760-a136-401e-8e16-4ade2bb103fd admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.001252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccdd760-a136-401e-8e16-4ade2bb103fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.003273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccdd760-a136-401e-8e16-4ade2bb103fd admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.006538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c40a85-a633-463f-bdd1-be6b7c1f8fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:37.007505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c40a85-a633-463f-bdd1-be6b7c1f8fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.010137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c40a85-a633-463f-bdd1-be6b7c1f8fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.010681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c40a85-a633-463f-bdd1-be6b7c1f8fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.013724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c40a85-a633-463f-bdd1-be6b7c1f8fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.018579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c40a85-a633-463f-bdd1-be6b7c1f8fe5 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.051449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:37.155424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:37.159108 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 937/1853] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:36 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:37.161123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdd760-a136-401e-8e16-4ade2bb103fd admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:37.166420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 917/1854] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:36 2022] POST /identity/v3/users => generated 292 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:37.166420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c40a85-a633-463f-bdd1-be6b7c1f8fe5 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:37.168206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6ccdd760-a136-401e-8e16-4ade2bb103fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:37.190550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c40a85-a633-463f-bdd1-be6b7c1f8fe5 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:37.201286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d756dfe-0e6b-494b-94f6-bf31c8ebcea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:37.203838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d756dfe-0e6b-494b-94f6-bf31c8ebcea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.204254 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d756dfe-0e6b-494b-94f6-bf31c8ebcea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.204551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d756dfe-0e6b-494b-94f6-bf31c8ebcea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.205518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d756dfe-0e6b-494b-94f6-bf31c8ebcea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.210915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d756dfe-0e6b-494b-94f6-bf31c8ebcea2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.233247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 918/1855] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:37.234548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d756dfe-0e6b-494b-94f6-bf31c8ebcea2 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:37.261276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d756dfe-0e6b-494b-94f6-bf31c8ebcea2 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:37.273835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8c137-7ccb-4ef6-a147-10d18d097679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:37.273835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8c137-7ccb-4ef6-a147-10d18d097679 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.273835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8c137-7ccb-4ef6-a147-10d18d097679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.273835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8c137-7ccb-4ef6-a147-10d18d097679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.274274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8c137-7ccb-4ef6-a147-10d18d097679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.274274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8c137-7ccb-4ef6-a147-10d18d097679 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.308519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 919/1856] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:37.308519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8c137-7ccb-4ef6-a147-10d18d097679 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:37.315419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdd760-a136-401e-8e16-4ade2bb103fd admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:37.322580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8c137-7ccb-4ef6-a147-10d18d097679 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:37.328403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de20ecf-554a-4b93-8c80-0955bccb2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:37.341734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de20ecf-554a-4b93-8c80-0955bccb2f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.342025 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de20ecf-554a-4b93-8c80-0955bccb2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.342300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de20ecf-554a-4b93-8c80-0955bccb2f33 admin admin] PATH_INFO: `/v3/projects/17cee16b7bac423dbae534454c1624f3/users/0981659d1ac34a13a8aaf8fe57c932f3/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.347871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761e5c2-9533-40a0-ae71-7acdb8f9cf92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:37.349914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761e5c2-9533-40a0-ae71-7acdb8f9cf92 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.350876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761e5c2-9533-40a0-ae71-7acdb8f9cf92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.353117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761e5c2-9533-40a0-ae71-7acdb8f9cf92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.366410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de20ecf-554a-4b93-8c80-0955bccb2f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17cee16b7bac423dbae534454c1624f3, user_id=0981659d1ac34a13a8aaf8fe57c932f3, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.370197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de20ecf-554a-4b93-8c80-0955bccb2f33 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.448802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 920/1857] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:37 2022] PUT /identity/v3/projects/17cee16b7bac423dbae534454c1624f3/users/0981659d1ac34a13a8aaf8fe57c932f3/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:37.452494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de20ecf-554a-4b93-8c80-0955bccb2f33 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:37.519003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761e5c2-9533-40a0-ae71-7acdb8f9cf92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.522552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761e5c2-9533-40a0-ae71-7acdb8f9cf92 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.569712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 938/1858] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:37.574073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761e5c2-9533-40a0-ae71-7acdb8f9cf92 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:37.604370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:37.613170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761e5c2-9533-40a0-ae71-7acdb8f9cf92 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:37.615417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de20ecf-554a-4b93-8c80-0955bccb2f33 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:37.630756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f0e8-ba6f-4035-ba62-25cc8402b883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:37.633498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f0e8-ba6f-4035-ba62-25cc8402b883 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.633890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92318aa-05a0-444d-8711-55bc1d285e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:37.634741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92318aa-05a0-444d-8711-55bc1d285e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.635204 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92318aa-05a0-444d-8711-55bc1d285e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.635959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92318aa-05a0-444d-8711-55bc1d285e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.636286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f0e8-ba6f-4035-ba62-25cc8402b883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.637342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f0e8-ba6f-4035-ba62-25cc8402b883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.638727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92318aa-05a0-444d-8711-55bc1d285e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.639669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f0e8-ba6f-4035-ba62-25cc8402b883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.644039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92318aa-05a0-444d-8711-55bc1d285e30 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.652652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f0e8-ba6f-4035-ba62-25cc8402b883 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.666970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 939/1859] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:37.668773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92318aa-05a0-444d-8711-55bc1d285e30 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:37.677363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 921/1860] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:37.681574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f0e8-ba6f-4035-ba62-25cc8402b883 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:37.702759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:37.703958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92318aa-05a0-444d-8711-55bc1d285e30 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:37.706264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f0e8-ba6f-4035-ba62-25cc8402b883 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:37.717538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f1202-8d91-4b73-8c17-a954084f3882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:37.718537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f1202-8d91-4b73-8c17-a954084f3882 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.718825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f1202-8d91-4b73-8c17-a954084f3882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.719111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f1202-8d91-4b73-8c17-a954084f3882 admin admin] PATH_INFO: `/v3/projects/e470e86966264a94bbc9cf5e97a00b52/users/8dde17bf982341899fbe77f7dd9b06a0/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.720647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6af34-a137-433d-9a45-4c4e3e26ec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:37.725354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6af34-a137-433d-9a45-4c4e3e26ec24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.728192 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6af34-a137-433d-9a45-4c4e3e26ec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.728634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6af34-a137-433d-9a45-4c4e3e26ec24 admin admin] PATH_INFO: `/v3/projects/17cee16b7bac423dbae534454c1624f3/users/0981659d1ac34a13a8aaf8fe57c932f3/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.732486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6af34-a137-433d-9a45-4c4e3e26ec24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17cee16b7bac423dbae534454c1624f3, user_id=0981659d1ac34a13a8aaf8fe57c932f3, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.735170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6af34-a137-433d-9a45-4c4e3e26ec24 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.748981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:37.761102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f1202-8d91-4b73-8c17-a954084f3882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e470e86966264a94bbc9cf5e97a00b52, user_id=8dde17bf982341899fbe77f7dd9b06a0, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:37.764084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f1202-8d91-4b73-8c17-a954084f3882 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:37.785961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 940/1861] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:37 2022] PUT /identity/v3/projects/17cee16b7bac423dbae534454c1624f3/users/0981659d1ac34a13a8aaf8fe57c932f3/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:37.788650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6af34-a137-433d-9a45-4c4e3e26ec24 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:37.797205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6cb6af34-a137-433d-9a45-4c4e3e26ec24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:37.832255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 922/1862] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:37 2022] PUT /identity/v3/projects/e470e86966264a94bbc9cf5e97a00b52/users/8dde17bf982341899fbe77f7dd9b06a0/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:37.832255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb06e0-12e9-4ccf-89e9-7e0c55abab1e None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:37.832255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb06e0-12e9-4ccf-89e9-7e0c55abab1e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:37.832255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb06e0-12e9-4ccf-89e9-7e0c55abab1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:37.915806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6af34-a137-433d-9a45-4c4e3e26ec24 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:37.926315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1ecb06e0-12e9-4ccf-89e9-7e0c55abab1e None None] MFA Rules not processed for user `0981659d1ac34a13a8aaf8fe57c932f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:37.928652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:37.930881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1ecb06e0-12e9-4ccf-89e9-7e0c55abab1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:38.011814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:38.014602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:38.014602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:38.014602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] PATH_INFO: `/v3/users/0327d85c4b364cceb5db7f5afab6019f` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:38.015149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0327d85c4b364cceb5db7f5afab6019f)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:38.017337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:38.127204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:38.135402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 923/1863] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:38.137122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0112f7c-9fcd-4d1a-bd77-e9ba1fd6fa17 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:38.137568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0112f7c-9fcd-4d1a-bd77-e9ba1fd6fa17 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:38.138383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0112f7c-9fcd-4d1a-bd77-e9ba1fd6fa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:38.226478 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b0112f7c-9fcd-4d1a-bd77-e9ba1fd6fa17 None None] MFA Rules not processed for user `8dde17bf982341899fbe77f7dd9b06a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:38.237827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b0112f7c-9fcd-4d1a-bd77-e9ba1fd6fa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:38.239417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0327d85c4b364cceb5db7f5afab6019f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:38.336318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0327d85c4b364cceb5db7f5afab6019f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:38.337100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0327d85c4b364cceb5db7f5afab6019f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:38.348440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0327d85c4b364cceb5db7f5afab6019f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:38.349180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0327d85c4b364cceb5db7f5afab6019f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:38.408103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 941/1864] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:37 2022] DELETE /identity/v3/users/0327d85c4b364cceb5db7f5afab6019f => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:38.408103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:38.408103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:38.410811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 924/1865] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:38.412212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0112f7c-9fcd-4d1a-bd77-e9ba1fd6fa17 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:38.414953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b0112f7c-9fcd-4d1a-bd77-e9ba1fd6fa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:38.445511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:38.525323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541b7b5-dfc9-4af4-88ce-2f7c4b21a56f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:38.530828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3ae7d-b8ad-46e7-baca-e73f3054a97b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:38.531948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3ae7d-b8ad-46e7-baca-e73f3054a97b service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:38.532917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3ae7d-b8ad-46e7-baca-e73f3054a97b service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:38.533460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3ae7d-b8ad-46e7-baca-e73f3054a97b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:38.535620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-cfe3ae7d-b8ad-46e7-baca-e73f3054a97b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:38.566063 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0112f7c-9fcd-4d1a-bd77-e9ba1fd6fa17 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:38.582648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:38.583368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:38.583696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:38.584023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:38.588494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:38.590620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:38.621676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:38.633821 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3ae7d-b8ad-46e7-baca-e73f3054a97b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:38.635477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3ae7d-b8ad-46e7-baca-e73f3054a97b service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:38.645595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 942/1866] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:38.648084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6fbcd-045e-4e80-8041-dce985a60bd6 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:38.648535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6fbcd-045e-4e80-8041-dce985a60bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:38.648968 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6fbcd-045e-4e80-8041-dce985a60bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:38.669640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-9eb6fbcd-045e-4e80-8041-dce985a60bd6 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:38.670858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9eb6fbcd-045e-4e80-8041-dce985a60bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:38.705902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:38.707729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:38.738925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 925/1867] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:38.746894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 943/1868] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:38.773405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:38.776540 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:38.831068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:38.891200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f0fcb-768c-4609-af7d-cdc21794c6e0 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:38.897617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99be84c-9ebc-40f2-a309-ac716543f847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:38.898523 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99be84c-9ebc-40f2-a309-ac716543f847 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:38.898961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99be84c-9ebc-40f2-a309-ac716543f847 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:38.899484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99be84c-9ebc-40f2-a309-ac716543f847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:38.912419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99be84c-9ebc-40f2-a309-ac716543f847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:38.923569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99be84c-9ebc-40f2-a309-ac716543f847 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:38.952098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 926/1869] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:38.962394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:39.265540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6fbcd-045e-4e80-8041-dce985a60bd6 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:39.288521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6fbcd-045e-4e80-8041-dce985a60bd6 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:39.302373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291cd5e-082a-4378-84fd-e3a534ff7f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:39.303911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291cd5e-082a-4378-84fd-e3a534ff7f4e service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:39.304197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291cd5e-082a-4378-84fd-e3a534ff7f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:39.304485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291cd5e-082a-4378-84fd-e3a534ff7f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:39.422000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:39.426453 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291cd5e-082a-4378-84fd-e3a534ff7f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:39.428836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291cd5e-082a-4378-84fd-e3a534ff7f4e service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:39.456867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 944/1870] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:39.468877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99be84c-9ebc-40f2-a309-ac716543f847 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:39.471658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f99be84c-9ebc-40f2-a309-ac716543f847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:39.489946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99be84c-9ebc-40f2-a309-ac716543f847 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:39.498059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:39.498966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:39.500617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:39.501156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] PATH_INFO: `/v3/users/8dde17bf982341899fbe77f7dd9b06a0` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:39.503455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dde17bf982341899fbe77f7dd9b06a0)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:39.505531 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:39.766341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:39.769096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dde17bf982341899fbe77f7dd9b06a0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:39.770075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dde17bf982341899fbe77f7dd9b06a0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:39.792118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dde17bf982341899fbe77f7dd9b06a0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:39.793183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dde17bf982341899fbe77f7dd9b06a0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:39.836801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dde17bf982341899fbe77f7dd9b06a0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:40.005904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:40.025856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 927/1871] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:39 2022] DELETE /identity/v3/users/8dde17bf982341899fbe77f7dd9b06a0 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:40.032212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02021d35-184c-4a2b-aa35-56931a11643a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:40.032748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02021d35-184c-4a2b-aa35-56931a11643a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:40.033470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02021d35-184c-4a2b-aa35-56931a11643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:40.075117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-02021d35-184c-4a2b-aa35-56931a11643a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:40.078445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-02021d35-184c-4a2b-aa35-56931a11643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:40.195978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 945/1872] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:40.223763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:40.230990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:40.354592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b145bb5-c306-488b-8ea5-168f03ca6e0c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:40.369842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02021d35-184c-4a2b-aa35-56931a11643a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:40.370914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044a556-a9e3-4ad1-b47b-36524069d5f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:40.371063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-02021d35-184c-4a2b-aa35-56931a11643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:40.372419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044a556-a9e3-4ad1-b47b-36524069d5f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:40.372923 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044a556-a9e3-4ad1-b47b-36524069d5f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:40.373415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044a556-a9e3-4ad1-b47b-36524069d5f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:40.385961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044a556-a9e3-4ad1-b47b-36524069d5f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:40.390751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044a556-a9e3-4ad1-b47b-36524069d5f3 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:40.399846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02021d35-184c-4a2b-aa35-56931a11643a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:40.404943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:40.421902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:40.426445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:40.426445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:40.426445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] PATH_INFO: `/v3/projects/4203bbdff4f14d9abb26c0fb343e1f73` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:40.426445 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4203bbdff4f14d9abb26c0fb343e1f73)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:40.427003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:40.432459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 928/1873] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:40.625595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4203bbdff4f14d9abb26c0fb343e1f73', 'request_id': 'req-968d07c3-297e-4080-b20e-7ac33a50fe6e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:40.709923 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4203bbdff4f14d9abb26c0fb343e1f73', 'request_id': 'req-968d07c3-297e-4080-b20e-7ac33a50fe6e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:40.719124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4203bbdff4f14d9abb26c0fb343e1f73', 'request_id': 'req-968d07c3-297e-4080-b20e-7ac33a50fe6e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:40.733309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] The token cache is being invalidate because project 4203bbdff4f14d9abb26c0fb343e1f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:40.734008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:40.735139 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:40.737959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 946/1874] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:40 2022] DELETE /identity/v3/projects/4203bbdff4f14d9abb26c0fb343e1f73 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:40.742794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044a556-a9e3-4ad1-b47b-36524069d5f3 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:40.756458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044a556-a9e3-4ad1-b47b-36524069d5f3 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:40.889105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:40.890069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:40.894298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:40.894298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] PATH_INFO: `/v3/users/d45038db2af04560b48b2156a7cc77a4` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:40.894298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d45038db2af04560b48b2156a7cc77a4)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:40.894550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:41.144416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd45038db2af04560b48b2156a7cc77a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:41.145536 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd45038db2af04560b48b2156a7cc77a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:41.158051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd45038db2af04560b48b2156a7cc77a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:41.160267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd45038db2af04560b48b2156a7cc77a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:41.202831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-95760f22-49d6-410c-9412-4dfb596727c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd45038db2af04560b48b2156a7cc77a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:41.327597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 929/1875] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:40 2022] DELETE /identity/v3/users/d45038db2af04560b48b2156a7cc77a4 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:41.391606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:41.394153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:41.445375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d07c3-297e-4080-b20e-7ac33a50fe6e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:41.519876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:41.520675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:41.520956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:41.521231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] PATH_INFO: `/v3/projects/e470e86966264a94bbc9cf5e97a00b52` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:41.523098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e470e86966264a94bbc9cf5e97a00b52)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:41.525088 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:41.622434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:41.965181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:41.981020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e470e86966264a94bbc9cf5e97a00b52', 'request_id': 'req-1721988b-1cd1-46eb-9581-fcf0661ae3e6'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.085760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e470e86966264a94bbc9cf5e97a00b52', 'request_id': 'req-1721988b-1cd1-46eb-9581-fcf0661ae3e6'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.091734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e470e86966264a94bbc9cf5e97a00b52', 'request_id': 'req-1721988b-1cd1-46eb-9581-fcf0661ae3e6'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.104799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] The token cache is being invalidate because project e470e86966264a94bbc9cf5e97a00b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:42.105147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.105752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.108185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 947/1876] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:41 2022] DELETE /identity/v3/projects/e470e86966264a94bbc9cf5e97a00b52 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:42.113051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d1abe-aded-4b66-aa2a-c5719784777c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.113348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d1abe-aded-4b66-aa2a-c5719784777c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.118467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d1abe-aded-4b66-aa2a-c5719784777c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.138700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-492d1abe-aded-4b66-aa2a-c5719784777c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:42.141895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-492d1abe-aded-4b66-aa2a-c5719784777c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.181394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:42.182831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.269214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 930/1877] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:42.277134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b6ff1-76e7-40ff-8530-a9d574dbac90 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.278363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b6ff1-76e7-40ff-8530-a9d574dbac90 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.278992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b6ff1-76e7-40ff-8530-a9d574dbac90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.321461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ef3b6ff1-76e7-40ff-8530-a9d574dbac90 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:42.322875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ef3b6ff1-76e7-40ff-8530-a9d574dbac90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.332558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721988b-1cd1-46eb-9581-fcf0661ae3e6 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:42.339567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:42.340455 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.340740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.341018 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] PATH_INFO: `/v3/projects/09380167a6fd4cdea1334201b55c9537` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.342464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09380167a6fd4cdea1334201b55c9537)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:42.343193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 931/1878] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:42.344939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:42.351206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b6ff1-76e7-40ff-8530-a9d574dbac90 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:42.363763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b6ff1-76e7-40ff-8530-a9d574dbac90 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:42.372605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:42.372940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.373343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.373633 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.375011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:42.375190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:42.377175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:42.399574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 932/1879] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:36:42.399574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:42.399574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.412016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29b77d-d32e-4fa4-b152-f311e1c5ef8b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:42.417034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:42.417886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.418171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.418447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] PATH_INFO: `/v3/users/d4aaefcce9914009af255400c9557875` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.419681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4aaefcce9914009af255400c9557875)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:42.421624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:42.446334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:42.481619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09380167a6fd4cdea1334201b55c9537', 'request_id': 'req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.548006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09380167a6fd4cdea1334201b55c9537', 'request_id': 'req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.554922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09380167a6fd4cdea1334201b55c9537', 'request_id': 'req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.566947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] The token cache is being invalidate because project 09380167a6fd4cdea1334201b55c9537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:42.567291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.568031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.570249 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 948/1880] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:42 2022] DELETE /identity/v3/projects/09380167a6fd4cdea1334201b55c9537 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:42.571049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:42.573852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.603289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4aaefcce9914009af255400c9557875'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.604271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4aaefcce9914009af255400c9557875'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.612197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4aaefcce9914009af255400c9557875'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.612891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4aaefcce9914009af255400c9557875'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.639617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4f8fcf2f-55bf-412c-949a-b290bf2419c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4aaefcce9914009af255400c9557875'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:42.644357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290dae9-4c77-4fbf-a5e0-9896d5a15c82 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:42.649268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162cff9-22b3-420e-b54a-b6ee6f89e827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:42.655158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162cff9-22b3-420e-b54a-b6ee6f89e827 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.655158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162cff9-22b3-420e-b54a-b6ee6f89e827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.655158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162cff9-22b3-420e-b54a-b6ee6f89e827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.655158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162cff9-22b3-420e-b54a-b6ee6f89e827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:42.655158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162cff9-22b3-420e-b54a-b6ee6f89e827 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:42.668604 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:42.713924 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 933/1881] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:42 2022] DELETE /identity/v3/users/d4aaefcce9914009af255400c9557875 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:42.715186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0db9-23a2-4670-96b8-c2aeb8b93913 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.715677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0db9-23a2-4670-96b8-c2aeb8b93913 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.715963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0db9-23a2-4670-96b8-c2aeb8b93913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.735086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-529a0db9-23a2-4670-96b8-c2aeb8b93913 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:42.736366 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-529a0db9-23a2-4670-96b8-c2aeb8b93913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.756596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 949/1882] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:42 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:42.758182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d725c-1e35-48c8-892f-415ebc710887 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.758473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d725c-1e35-48c8-892f-415ebc710887 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.758890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d725c-1e35-48c8-892f-415ebc710887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.778663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-af0d725c-1e35-48c8-892f-415ebc710887 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:42.780165 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-af0d725c-1e35-48c8-892f-415ebc710887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.815337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 934/1883] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:42.816354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a0db9-23a2-4670-96b8-c2aeb8b93913 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:42.827862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-529a0db9-23a2-4670-96b8-c2aeb8b93913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.829967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a0db9-23a2-4670-96b8-c2aeb8b93913 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:42.841548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 950/1884] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:42.842931 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aebbe-5eb4-4eee-acad-757619140387 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.843138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aebbe-5eb4-4eee-acad-757619140387 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.843419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aebbe-5eb4-4eee-acad-757619140387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.862682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-bc6aebbe-5eb4-4eee-acad-757619140387 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:42.864089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-bc6aebbe-5eb4-4eee-acad-757619140387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.872898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 951/1885] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:42.873799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6aebbe-5eb4-4eee-acad-757619140387 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:42.875087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-bc6aebbe-5eb4-4eee-acad-757619140387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:42.890334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c7bac-46cd-4a89-8f25-4fbc4ea22ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:42.891111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c7bac-46cd-4a89-8f25-4fbc4ea22ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.891405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c7bac-46cd-4a89-8f25-4fbc4ea22ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.891724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c7bac-46cd-4a89-8f25-4fbc4ea22ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.892460 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c7bac-46cd-4a89-8f25-4fbc4ea22ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:42.894491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c7bac-46cd-4a89-8f25-4fbc4ea22ad5 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:42.931660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6aebbe-5eb4-4eee-acad-757619140387 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:42.935329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079389d-bc4b-4b78-b684-97ba1441dd44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:42.936008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079389d-bc4b-4b78-b684-97ba1441dd44 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.936287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079389d-bc4b-4b78-b684-97ba1441dd44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.936562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079389d-bc4b-4b78-b684-97ba1441dd44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.945677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079389d-bc4b-4b78-b684-97ba1441dd44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:42.947226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079389d-bc4b-4b78-b684-97ba1441dd44 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:42.956646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 952/1886] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:42.957473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079389d-bc4b-4b78-b684-97ba1441dd44 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:42.966887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079389d-bc4b-4b78-b684-97ba1441dd44 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:42.971843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f9af8-0166-4e71-9fb9-dd79609a0add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:42.975725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f9af8-0166-4e71-9fb9-dd79609a0add admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.975851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 935/1887] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:42 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:42.976289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f9af8-0166-4e71-9fb9-dd79609a0add admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.976289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f9af8-0166-4e71-9fb9-dd79609a0add admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.976289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f9af8-0166-4e71-9fb9-dd79609a0add admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:42.976289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f9af8-0166-4e71-9fb9-dd79609a0add admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:42.976610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f9af8-0166-4e71-9fb9-dd79609a0add admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:42.977232 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c7bac-46cd-4a89-8f25-4fbc4ea22ad5 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:42.988494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c7bac-46cd-4a89-8f25-4fbc4ea22ad5 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:42.991579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 953/1888] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:42.995265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42385e63-9278-4c2d-942d-3503f8b830a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:42.995976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42385e63-9278-4c2d-942d-3503f8b830a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:42.997468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42385e63-9278-4c2d-942d-3503f8b830a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:42.997468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42385e63-9278-4c2d-942d-3503f8b830a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:42.997468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42385e63-9278-4c2d-942d-3503f8b830a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:42.998925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42385e63-9278-4c2d-942d-3503f8b830a0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.001196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f9af8-0166-4e71-9fb9-dd79609a0add admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.009635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 936/1889] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:43.012709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f9af8-0166-4e71-9fb9-dd79609a0add admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.014349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42385e63-9278-4c2d-942d-3503f8b830a0 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.017677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631a568-37c6-4790-9c4d-01b9f1a4f943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.018342 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631a568-37c6-4790-9c4d-01b9f1a4f943 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.018789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631a568-37c6-4790-9c4d-01b9f1a4f943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.019073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631a568-37c6-4790-9c4d-01b9f1a4f943 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.019801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631a568-37c6-4790-9c4d-01b9f1a4f943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.021842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631a568-37c6-4790-9c4d-01b9f1a4f943 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.026202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42385e63-9278-4c2d-942d-3503f8b830a0 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.031598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891a3fa-9122-4797-a73b-b06ebdff5e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.032510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891a3fa-9122-4797-a73b-b06ebdff5e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.032801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891a3fa-9122-4797-a73b-b06ebdff5e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.033085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891a3fa-9122-4797-a73b-b06ebdff5e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.033988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891a3fa-9122-4797-a73b-b06ebdff5e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.035639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891a3fa-9122-4797-a73b-b06ebdff5e85 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.048065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 937/1890] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:43.052325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891a3fa-9122-4797-a73b-b06ebdff5e85 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.063066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891a3fa-9122-4797-a73b-b06ebdff5e85 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.067844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484cad4-f554-4598-8d74-0b703338db3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.068745 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484cad4-f554-4598-8d74-0b703338db3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.069024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484cad4-f554-4598-8d74-0b703338db3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.069294 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484cad4-f554-4598-8d74-0b703338db3e admin admin] PATH_INFO: `/v3/projects/0b6930878a5c4f9b847362662c4d819b/users/5a7eb45bf2c94759b3926b94f10c65b0/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.083238 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484cad4-f554-4598-8d74-0b703338db3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6930878a5c4f9b847362662c4d819b, user_id=5a7eb45bf2c94759b3926b94f10c65b0, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.086479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484cad4-f554-4598-8d74-0b703338db3e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.103691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 954/1891] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:42 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:43.109321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631a568-37c6-4790-9c4d-01b9f1a4f943 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.169946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631a568-37c6-4790-9c4d-01b9f1a4f943 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.191198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 938/1892] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:43 2022] PUT /identity/v3/projects/0b6930878a5c4f9b847362662c4d819b/users/5a7eb45bf2c94759b3926b94f10c65b0/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:43.194123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917c0e3-407a-4aa8-866f-15e16940cf41 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.194870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917c0e3-407a-4aa8-866f-15e16940cf41 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.195601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917c0e3-407a-4aa8-866f-15e16940cf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.224536 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e917c0e3-407a-4aa8-866f-15e16940cf41 None None] MFA Rules not processed for user `5a7eb45bf2c94759b3926b94f10c65b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:43.226463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e917c0e3-407a-4aa8-866f-15e16940cf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:43.231427 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00726fc-a7e4-4730-aaad-d52747dd75a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.232269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00726fc-a7e4-4730-aaad-d52747dd75a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.232269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00726fc-a7e4-4730-aaad-d52747dd75a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.232652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00726fc-a7e4-4730-aaad-d52747dd75a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.233436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00726fc-a7e4-4730-aaad-d52747dd75a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.235569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00726fc-a7e4-4730-aaad-d52747dd75a7 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.323658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 939/1893] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:43.323787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 955/1894] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:43 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:43.327792 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917c0e3-407a-4aa8-866f-15e16940cf41 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.333393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00726fc-a7e4-4730-aaad-d52747dd75a7 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.337466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d00726fc-a7e4-4730-aaad-d52747dd75a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:43.339895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917c0e3-407a-4aa8-866f-15e16940cf41 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.347395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624c9c9-abfd-4905-8888-730ee551fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.348077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c9c9-abfd-4905-8888-730ee551fd86 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.348369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c9c9-abfd-4905-8888-730ee551fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.348648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c9c9-abfd-4905-8888-730ee551fd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.349590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624c9c9-abfd-4905-8888-730ee551fd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.354169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624c9c9-abfd-4905-8888-730ee551fd86 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.367662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 940/1895] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:43.371636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624c9c9-abfd-4905-8888-730ee551fd86 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.383387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624c9c9-abfd-4905-8888-730ee551fd86 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.390654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4c88b-9a6f-4da3-bea8-b5221d057b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.393126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4c88b-9a6f-4da3-bea8-b5221d057b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.393422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4c88b-9a6f-4da3-bea8-b5221d057b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.393707 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4c88b-9a6f-4da3-bea8-b5221d057b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.394449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4c88b-9a6f-4da3-bea8-b5221d057b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.396170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4c88b-9a6f-4da3-bea8-b5221d057b86 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.400892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:43.403506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00726fc-a7e4-4730-aaad-d52747dd75a7 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.410536 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2b6ad-f656-4af4-9a6d-30a87b510007 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e33d1473077247cc97dcfd99b3da5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.411306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2b6ad-f656-4af4-9a6d-30a87b510007 service glance] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.411522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2b6ad-f656-4af4-9a6d-30a87b510007 service glance] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.411851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2b6ad-f656-4af4-9a6d-30a87b510007 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.421611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 941/1896] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:43.424965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4c88b-9a6f-4da3-bea8-b5221d057b86 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.424965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4ce4c88b-9a6f-4da3-bea8-b5221d057b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:43.446040 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2b6ad-f656-4af4-9a6d-30a87b510007 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.446768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4c88b-9a6f-4da3-bea8-b5221d057b86 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.447070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2b6ad-f656-4af4-9a6d-30a87b510007 service glance] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.447149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.447236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.447676 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.447924 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] PATH_INFO: `/v3/projects/b41ab9ca798f42d0be421d7e02ab694e` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.449455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41ab9ca798f42d0be421d7e02ab694e)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.451470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.454728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 956/1897] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:43.455781 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2b6ad-f656-4af4-9a6d-30a87b510007 service glance] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.469760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2b6ad-f656-4af4-9a6d-30a87b510007 service glance] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.480222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af44304-57f3-4921-89ca-435f0da4d641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.481146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af44304-57f3-4921-89ca-435f0da4d641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.481430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af44304-57f3-4921-89ca-435f0da4d641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.481769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af44304-57f3-4921-89ca-435f0da4d641 admin admin] PATH_INFO: `/v3/projects/28633dba4a3d4102ae7539ab309525c3/users/8e1aebc412944392ae6edb0373c3f4f9/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.503653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af44304-57f3-4921-89ca-435f0da4d641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28633dba4a3d4102ae7539ab309525c3, user_id=8e1aebc412944392ae6edb0373c3f4f9, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.507620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af44304-57f3-4921-89ca-435f0da4d641 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.594322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:43.611988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 957/1898] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:43 2022] PUT /identity/v3/projects/28633dba4a3d4102ae7539ab309525c3/users/8e1aebc412944392ae6edb0373c3f4f9/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:43.612089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41ab9ca798f42d0be421d7e02ab694e', 'request_id': 'req-de1afaf7-504d-408d-99f4-218a6ed50cae'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:43.612999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c48799-a7b9-41b1-9203-e3b2e00ba789 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.613420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c48799-a7b9-41b1-9203-e3b2e00ba789 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.614084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c48799-a7b9-41b1-9203-e3b2e00ba789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.625698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41ab9ca798f42d0be421d7e02ab694e', 'request_id': 'req-de1afaf7-504d-408d-99f4-218a6ed50cae'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:43.635699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41ab9ca798f42d0be421d7e02ab694e', 'request_id': 'req-de1afaf7-504d-408d-99f4-218a6ed50cae'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:43.662563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-97c48799-a7b9-41b1-9203-e3b2e00ba789 None None] MFA Rules not processed for user `8e1aebc412944392ae6edb0373c3f4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:43.664069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-97c48799-a7b9-41b1-9203-e3b2e00ba789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:43.714311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] The token cache is being invalidate because project b41ab9ca798f42d0be421d7e02ab694e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:43.714779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:43.715493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:43.717404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 942/1899] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:43 2022] DELETE /identity/v3/projects/b41ab9ca798f42d0be421d7e02ab694e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:43.719706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.719706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:43.738997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 958/1900] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:43.742242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c48799-a7b9-41b1-9203-e3b2e00ba789 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:43.743862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-97c48799-a7b9-41b1-9203-e3b2e00ba789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:43.801143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1afaf7-504d-408d-99f4-218a6ed50cae admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.839362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c48799-a7b9-41b1-9203-e3b2e00ba789 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:43.846828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.848112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.848938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.849219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] PATH_INFO: `/v3/users/0981659d1ac34a13a8aaf8fe57c932f3` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.851442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0981659d1ac34a13a8aaf8fe57c932f3)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.855050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.868610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:43.869498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:43.869783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:43.870062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] PATH_INFO: `/v3/users/3e018353a3b740a99efcf47b6f59b909` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:43.871288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e018353a3b740a99efcf47b6f59b909)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:43.873565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:43.886776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:44.033582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:44.069327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0981659d1ac34a13a8aaf8fe57c932f3'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.137983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0981659d1ac34a13a8aaf8fe57c932f3'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.138110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0981659d1ac34a13a8aaf8fe57c932f3'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.141155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e018353a3b740a99efcf47b6f59b909'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.142056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e018353a3b740a99efcf47b6f59b909'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.145943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0981659d1ac34a13a8aaf8fe57c932f3'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.146679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0981659d1ac34a13a8aaf8fe57c932f3'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.149453 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e018353a3b740a99efcf47b6f59b909'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.150109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e018353a3b740a99efcf47b6f59b909'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.162495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-602bc151-375d-4f10-898d-847a4fac456f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e018353a3b740a99efcf47b6f59b909'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.164915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 959/1901] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:43 2022] DELETE /identity/v3/users/0981659d1ac34a13a8aaf8fe57c932f3 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:44.165723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:44.167084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:44.236428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 943/1902] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:43 2022] DELETE /identity/v3/users/3e018353a3b740a99efcf47b6f59b909 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:44.238065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f95e9e-3a88-4e4d-bb45-1fba39b7f1cc admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:44.241342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6ce36-b81b-4510-97af-e74bf2d3abdc None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.241705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6ce36-b81b-4510-97af-e74bf2d3abdc None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.242387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6ce36-b81b-4510-97af-e74bf2d3abdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:44.260636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-29e6ce36-b81b-4510-97af-e74bf2d3abdc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:44.266358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-29e6ce36-b81b-4510-97af-e74bf2d3abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:44.301106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe93eb-f8a9-4ff3-92b5-d1126812c380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:44.301776 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe93eb-f8a9-4ff3-92b5-d1126812c380 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.302062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe93eb-f8a9-4ff3-92b5-d1126812c380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.302339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe93eb-f8a9-4ff3-92b5-d1126812c380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:44.303590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe93eb-f8a9-4ff3-92b5-d1126812c380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:44.305544 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe93eb-f8a9-4ff3-92b5-d1126812c380 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:44.315365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:44.322313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 944/1903] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:44.331966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6ce36-b81b-4510-97af-e74bf2d3abdc None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:44.333324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-29e6ce36-b81b-4510-97af-e74bf2d3abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:44.355522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:44.404646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 960/1904] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:44 2022] POST /identity/v3/projects => generated 387 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:44.407500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe93eb-f8a9-4ff3-92b5-d1126812c380 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:44.409252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-16fe93eb-f8a9-4ff3-92b5-d1126812c380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:44.423941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6ce36-b81b-4510-97af-e74bf2d3abdc None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:44.483050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196241b-cbc4-4342-9bb7-ce92e07eebab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:44.483847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196241b-cbc4-4342-9bb7-ce92e07eebab service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.484197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196241b-cbc4-4342-9bb7-ce92e07eebab service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.484608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196241b-cbc4-4342-9bb7-ce92e07eebab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:44.493918 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe93eb-f8a9-4ff3-92b5-d1126812c380 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:44.497184 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196241b-cbc4-4342-9bb7-ce92e07eebab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:44.498636 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196241b-cbc4-4342-9bb7-ce92e07eebab service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:44.505356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61019f5-47e2-48f8-b2b2-10a18dc3e847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:44.509095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61019f5-47e2-48f8-b2b2-10a18dc3e847 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.509095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61019f5-47e2-48f8-b2b2-10a18dc3e847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.509095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61019f5-47e2-48f8-b2b2-10a18dc3e847 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:44.509095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61019f5-47e2-48f8-b2b2-10a18dc3e847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:44.510481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 945/1905] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:36:44.513020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61019f5-47e2-48f8-b2b2-10a18dc3e847 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:44.599565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 961/1906] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:44 2022] POST /identity/v3/users => generated 300 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:44.604136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196241b-cbc4-4342-9bb7-ce92e07eebab service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:44.613774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196241b-cbc4-4342-9bb7-ce92e07eebab service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:44.618634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0a020-8015-4b84-98fd-147d6e08db78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:44.619290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0a020-8015-4b84-98fd-147d6e08db78 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.619605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0a020-8015-4b84-98fd-147d6e08db78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.623968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0a020-8015-4b84-98fd-147d6e08db78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:44.623968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0a020-8015-4b84-98fd-147d6e08db78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:44.623968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0a020-8015-4b84-98fd-147d6e08db78 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:44.632867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 946/1907] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:44.636988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61019f5-47e2-48f8-b2b2-10a18dc3e847 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:44.650869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0a020-8015-4b84-98fd-147d6e08db78 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:44.652696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-90a0a020-8015-4b84-98fd-147d6e08db78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:44.656989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61019f5-47e2-48f8-b2b2-10a18dc3e847 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:44.668167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:44.669473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e05ab5-49a2-4d5e-a0d6-49bc67ccc742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:44.670872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e05ab5-49a2-4d5e-a0d6-49bc67ccc742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.671301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e05ab5-49a2-4d5e-a0d6-49bc67ccc742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.671769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e05ab5-49a2-4d5e-a0d6-49bc67ccc742 admin admin] PATH_INFO: `/v3/projects/5d0f00eeb4924a40a6fb7acdef0caad0/users/48ad6d7429f24eff9000d01a5cc3ae98/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:44.703383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e05ab5-49a2-4d5e-a0d6-49bc67ccc742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d0f00eeb4924a40a6fb7acdef0caad0, user_id=48ad6d7429f24eff9000d01a5cc3ae98, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:44.705311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0a020-8015-4b84-98fd-147d6e08db78 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:44.706244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e05ab5-49a2-4d5e-a0d6-49bc67ccc742 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:44.721438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:44.722324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.722602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.722877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] PATH_INFO: `/v3/projects/17cee16b7bac423dbae534454c1624f3` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:44.727126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] RBAC: Authorizing `identity:delete_project(project_id=17cee16b7bac423dbae534454c1624f3)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:44.731252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:44.822334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 962/1908] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:44 2022] PUT /identity/v3/projects/5d0f00eeb4924a40a6fb7acdef0caad0/users/48ad6d7429f24eff9000d01a5cc3ae98/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:44.827794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e05ab5-49a2-4d5e-a0d6-49bc67ccc742 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:44.867288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17cee16b7bac423dbae534454c1624f3', 'request_id': 'req-0ca01b08-aa8c-46a6-ad52-61f5238440de'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.876462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17cee16b7bac423dbae534454c1624f3', 'request_id': 'req-0ca01b08-aa8c-46a6-ad52-61f5238440de'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.886002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17cee16b7bac423dbae534454c1624f3', 'request_id': 'req-0ca01b08-aa8c-46a6-ad52-61f5238440de'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.903207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e05ab5-49a2-4d5e-a0d6-49bc67ccc742 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:44.962506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c31a3-e52a-4908-a2a7-f5fcb9ebd1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:44.962506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c31a3-e52a-4908-a2a7-f5fcb9ebd1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.962506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c31a3-e52a-4908-a2a7-f5fcb9ebd1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.962832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c31a3-e52a-4908-a2a7-f5fcb9ebd1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:44.963294 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c31a3-e52a-4908-a2a7-f5fcb9ebd1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:44.965097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c31a3-e52a-4908-a2a7-f5fcb9ebd1ff admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:44.976434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 963/1909] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:44.980344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c31a3-e52a-4908-a2a7-f5fcb9ebd1ff admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:44.987873 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] The token cache is being invalidate because project 17cee16b7bac423dbae534454c1624f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:44.988167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.989072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ca01b08-aa8c-46a6-ad52-61f5238440de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:44.991377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c31a3-e52a-4908-a2a7-f5fcb9ebd1ff admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:44.994641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 947/1910] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:44 2022] DELETE /identity/v3/projects/17cee16b7bac423dbae534454c1624f3 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:44.998409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f64ca9-8e11-4221-bed2-890001e9bd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:44.999319 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64ca9-8e11-4221-bed2-890001e9bd7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.999444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0a8cc-8101-4d1f-b4b2-07a5bc77ef89 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:44.999631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64ca9-8e11-4221-bed2-890001e9bd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.999752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0a8cc-8101-4d1f-b4b2-07a5bc77ef89 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:44.999911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64ca9-8e11-4221-bed2-890001e9bd7c admin admin] PATH_INFO: `/v3/projects/5d0f00eeb4924a40a6fb7acdef0caad0/users/48ad6d7429f24eff9000d01a5cc3ae98/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.000008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0a8cc-8101-4d1f-b4b2-07a5bc77ef89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.002005 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f64ca9-8e11-4221-bed2-890001e9bd7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d0f00eeb4924a40a6fb7acdef0caad0, user_id=48ad6d7429f24eff9000d01a5cc3ae98, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:45.004788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f64ca9-8e11-4221-bed2-890001e9bd7c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:45.018397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b2b0a8cc-8101-4d1f-b4b2-07a5bc77ef89 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:45.019655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b2b0a8cc-8101-4d1f-b4b2-07a5bc77ef89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:45.031627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 964/1911] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:44 2022] PUT /identity/v3/projects/5d0f00eeb4924a40a6fb7acdef0caad0/users/48ad6d7429f24eff9000d01a5cc3ae98/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:45.037351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856a28a-fb54-4141-9e7f-2f08bdfe582e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:45.037730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856a28a-fb54-4141-9e7f-2f08bdfe582e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:45.038002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856a28a-fb54-4141-9e7f-2f08bdfe582e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.065803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7856a28a-fb54-4141-9e7f-2f08bdfe582e None None] MFA Rules not processed for user `48ad6d7429f24eff9000d01a5cc3ae98`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:45.067024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7856a28a-fb54-4141-9e7f-2f08bdfe582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:45.087758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 948/1912] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:45.095315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a800f-8a0e-4445-a472-b2aeb9cb3eb7 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:45.095628 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a800f-8a0e-4445-a472-b2aeb9cb3eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:45.095913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a800f-8a0e-4445-a472-b2aeb9cb3eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.116461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ad4a800f-8a0e-4445-a472-b2aeb9cb3eb7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:45.117685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ad4a800f-8a0e-4445-a472-b2aeb9cb3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:45.126801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 949/1913] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:45.134553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a800f-8a0e-4445-a472-b2aeb9cb3eb7 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:45.142910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 965/1914] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:45.143779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856a28a-fb54-4141-9e7f-2f08bdfe582e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:45.146244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7856a28a-fb54-4141-9e7f-2f08bdfe582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:45.150327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:45.154197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a800f-8a0e-4445-a472-b2aeb9cb3eb7 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:45.163456 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:45.164364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:45.164996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:45.165525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.167150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:45.168504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:45.169277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856a28a-fb54-4141-9e7f-2f08bdfe582e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:45.171833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:45.174180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:45.175153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:45.175434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:45.180110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] PATH_INFO: `/v3/projects/f07300134844484a950ecf052af5ecf7` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.180110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f07300134844484a950ecf052af5ecf7)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:45.180110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:45.194482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 950/1915] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:45.195379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:45.200751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:45.262318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c487e5-b7ba-4f5d-9195-98a1c82a5c8e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:45.266063 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3183e-b186-4abc-ae7b-41ebc5dd1cdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:45.267228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3183e-b186-4abc-ae7b-41ebc5dd1cdf service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:45.267516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3183e-b186-4abc-ae7b-41ebc5dd1cdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:45.267921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3183e-b186-4abc-ae7b-41ebc5dd1cdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.283073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3183e-b186-4abc-ae7b-41ebc5dd1cdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:45.284655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3183e-b186-4abc-ae7b-41ebc5dd1cdf service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:45.286641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f07300134844484a950ecf052af5ecf7', 'request_id': 'req-23ba23c2-55e9-441c-9c9a-69586efb3247'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.295440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 951/1916] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:45.300700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3183e-b186-4abc-ae7b-41ebc5dd1cdf service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:45.307071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3183e-b186-4abc-ae7b-41ebc5dd1cdf service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:45.353360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f07300134844484a950ecf052af5ecf7', 'request_id': 'req-23ba23c2-55e9-441c-9c9a-69586efb3247'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.359406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f07300134844484a950ecf052af5ecf7', 'request_id': 'req-23ba23c2-55e9-441c-9c9a-69586efb3247'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.367817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e273b9-eb63-4b1b-92ea-bd0b06d1fa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:45.368486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e273b9-eb63-4b1b-92ea-bd0b06d1fa43 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:45.368814 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e273b9-eb63-4b1b-92ea-bd0b06d1fa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:45.369123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e273b9-eb63-4b1b-92ea-bd0b06d1fa43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.369882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e273b9-eb63-4b1b-92ea-bd0b06d1fa43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:45.370668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] The token cache is being invalidate because project f07300134844484a950ecf052af5ecf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:45.371004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.371830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e273b9-eb63-4b1b-92ea-bd0b06d1fa43 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:45.371994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.373474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:45.374755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 966/1917] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:45 2022] DELETE /identity/v3/projects/f07300134844484a950ecf052af5ecf7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:45.375926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:45.377340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:45.389825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba23c2-55e9-441c-9c9a-69586efb3247 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:45.394487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:45.395292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:45.395608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:45.395888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] PATH_INFO: `/v3/users/8e1aebc412944392ae6edb0373c3f4f9` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.399782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e1aebc412944392ae6edb0373c3f4f9)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:45.399782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:45.443562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 952/1918] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:45 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:45.443562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e273b9-eb63-4b1b-92ea-bd0b06d1fa43 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:45.443562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-48e273b9-eb63-4b1b-92ea-bd0b06d1fa43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:45.521197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e273b9-eb63-4b1b-92ea-bd0b06d1fa43 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:45.531019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:45.532093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:45.532485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:45.532885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] PATH_INFO: `/v3/users/bdbdd503440a4cd29a46a3a56be2bfff` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.534239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdbdd503440a4cd29a46a3a56be2bfff)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:45.536323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:45.613742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e1aebc412944392ae6edb0373c3f4f9'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.622639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:45.673558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e1aebc412944392ae6edb0373c3f4f9'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.674161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e1aebc412944392ae6edb0373c3f4f9'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.682660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e1aebc412944392ae6edb0373c3f4f9'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.683324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e1aebc412944392ae6edb0373c3f4f9'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.704559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 967/1919] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:45 2022] DELETE /identity/v3/users/8e1aebc412944392ae6edb0373c3f4f9 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:45.708295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:45.710480 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:45.716134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdbdd503440a4cd29a46a3a56be2bfff'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.716134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdbdd503440a4cd29a46a3a56be2bfff'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.722553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdbdd503440a4cd29a46a3a56be2bfff'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.723337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdbdd503440a4cd29a46a3a56be2bfff'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.732517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:45.743202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c48c8e6e-44e3-4b22-bdd6-01074b7ec7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdbdd503440a4cd29a46a3a56be2bfff'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:45.797536 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb69a76-c55c-4c44-919c-8919ed8726a0 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:45.809282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29537f6-5622-4df3-865c-6915d380580f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:45.812025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29537f6-5622-4df3-865c-6915d380580f admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:45.812025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29537f6-5622-4df3-865c-6915d380580f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:45.812025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29537f6-5622-4df3-865c-6915d380580f admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:45.812025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29537f6-5622-4df3-865c-6915d380580f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:45.821730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29537f6-5622-4df3-865c-6915d380580f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:46.336205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 953/1920] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:45 2022] DELETE /identity/v3/users/bdbdd503440a4cd29a46a3a56be2bfff => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:46.337731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22043582-8692-4a48-99f6-a7162607b516 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:46.338728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22043582-8692-4a48-99f6-a7162607b516 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:46.338728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22043582-8692-4a48-99f6-a7162607b516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:46.356405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-22043582-8692-4a48-99f6-a7162607b516 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:46.357750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-22043582-8692-4a48-99f6-a7162607b516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:46.433738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 954/1921] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:46.447162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22043582-8692-4a48-99f6-a7162607b516 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:46.448775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-22043582-8692-4a48-99f6-a7162607b516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:46.515870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22043582-8692-4a48-99f6-a7162607b516 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:46.520922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b61c7-8dd3-4d1b-a03e-4588df6974a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:46.521748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b61c7-8dd3-4d1b-a03e-4588df6974a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:46.522346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b61c7-8dd3-4d1b-a03e-4588df6974a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:46.522508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b61c7-8dd3-4d1b-a03e-4588df6974a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:46.532544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b61c7-8dd3-4d1b-a03e-4588df6974a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:46.533987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b61c7-8dd3-4d1b-a03e-4588df6974a7 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:46.544248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 955/1922] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:36:46.940199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 968/1923] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:45 2022] POST /identity/v3/users => generated 287 bytes in 1232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:46.941428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b61c7-8dd3-4d1b-a03e-4588df6974a7 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:46.942861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d49b61c7-8dd3-4d1b-a03e-4588df6974a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:46.978148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b61c7-8dd3-4d1b-a03e-4588df6974a7 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:46.985465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85aa9a-3c2e-45bd-8fef-b3be85901c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:46.986242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85aa9a-3c2e-45bd-8fef-b3be85901c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:46.986537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85aa9a-3c2e-45bd-8fef-b3be85901c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:46.986817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85aa9a-3c2e-45bd-8fef-b3be85901c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:46.988102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85aa9a-3c2e-45bd-8fef-b3be85901c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:46.990260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85aa9a-3c2e-45bd-8fef-b3be85901c3f admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:47.002278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 956/1924] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:47.005646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29537f6-5622-4df3-865c-6915d380580f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:47.020915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29537f6-5622-4df3-865c-6915d380580f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:47.027324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d26f35-97d2-43d8-836f-8b8f0dda4ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:47.028283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d26f35-97d2-43d8-836f-8b8f0dda4ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:47.028565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d26f35-97d2-43d8-836f-8b8f0dda4ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:47.029003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d26f35-97d2-43d8-836f-8b8f0dda4ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:47.029942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d26f35-97d2-43d8-836f-8b8f0dda4ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:47.031734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d26f35-97d2-43d8-836f-8b8f0dda4ad5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:47.051229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85aa9a-3c2e-45bd-8fef-b3be85901c3f admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:47.059196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 969/1925] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:47.066190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85aa9a-3c2e-45bd-8fef-b3be85901c3f admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:47.076460 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c89daf-cae3-4f13-8098-3a4744b05882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:47.078684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c89daf-cae3-4f13-8098-3a4744b05882 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:47.079215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c89daf-cae3-4f13-8098-3a4744b05882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:47.079716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c89daf-cae3-4f13-8098-3a4744b05882 admin admin] PATH_INFO: `/v3/projects/5194ccc9cb9e4a79820a1fa134b32726/users/ecd3ddc472d54a0b981707a3497c0255/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:47.105453 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c89daf-cae3-4f13-8098-3a4744b05882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5194ccc9cb9e4a79820a1fa134b32726, user_id=ecd3ddc472d54a0b981707a3497c0255, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:47.108511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c89daf-cae3-4f13-8098-3a4744b05882 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:47.199701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bb7c0-d8dd-47ee-a63d-2eb6fbf854cd None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:47.200635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bb7c0-d8dd-47ee-a63d-2eb6fbf854cd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:47.201040 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bb7c0-d8dd-47ee-a63d-2eb6fbf854cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:47.203902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 957/1926] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:47 2022] PUT /identity/v3/projects/5194ccc9cb9e4a79820a1fa134b32726/users/ecd3ddc472d54a0b981707a3497c0255/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:47.246585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7c1bb7c0-d8dd-47ee-a63d-2eb6fbf854cd None None] MFA Rules not processed for user `ecd3ddc472d54a0b981707a3497c0255`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:47.247985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7c1bb7c0-d8dd-47ee-a63d-2eb6fbf854cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:47.302570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:47.331189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 970/1927] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:47.350299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c89daf-cae3-4f13-8098-3a4744b05882 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:47.352595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b6c89daf-cae3-4f13-8098-3a4744b05882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:47.370497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bb7c0-d8dd-47ee-a63d-2eb6fbf854cd None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:47.372555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7c1bb7c0-d8dd-47ee-a63d-2eb6fbf854cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:47.435198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c89daf-cae3-4f13-8098-3a4744b05882 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:47.446552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489bbe2-8b25-472b-b03c-ac1cbe8c94b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:47.458841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489bbe2-8b25-472b-b03c-ac1cbe8c94b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:47.459068 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489bbe2-8b25-472b-b03c-ac1cbe8c94b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:47.464829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489bbe2-8b25-472b-b03c-ac1cbe8c94b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:47.473649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489bbe2-8b25-472b-b03c-ac1cbe8c94b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:47.475385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489bbe2-8b25-472b-b03c-ac1cbe8c94b6 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:47.485099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bb7c0-d8dd-47ee-a63d-2eb6fbf854cd None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:47.485277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 958/1928] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:47.489712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489bbe2-8b25-472b-b03c-ac1cbe8c94b6 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:47.492252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c489bbe2-8b25-472b-b03c-ac1cbe8c94b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:47.495344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:47.498302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:47.498724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:47.499018 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] PATH_INFO: `/v3/projects/13324525110c404ca7d9bab534f5e7ca` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:47.503680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13324525110c404ca7d9bab534f5e7ca)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:47.505844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:47.523576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:47.533506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489bbe2-8b25-472b-b03c-ac1cbe8c94b6 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:47.552316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:47.552316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:47.552316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:47.552316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] PATH_INFO: `/v3/projects/28633dba4a3d4102ae7539ab309525c3` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:47.552674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28633dba4a3d4102ae7539ab309525c3)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:47.554085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:47.687919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28633dba4a3d4102ae7539ab309525c3', 'request_id': 'req-729cd88c-92f4-4494-b3d2-4b7b6992b153'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.690829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13324525110c404ca7d9bab534f5e7ca', 'request_id': 'req-243ac94a-aa30-440e-bea0-0aa0d5074b96'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.705164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28633dba4a3d4102ae7539ab309525c3', 'request_id': 'req-729cd88c-92f4-4494-b3d2-4b7b6992b153'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.712185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28633dba4a3d4102ae7539ab309525c3', 'request_id': 'req-729cd88c-92f4-4494-b3d2-4b7b6992b153'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.751210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13324525110c404ca7d9bab534f5e7ca', 'request_id': 'req-243ac94a-aa30-440e-bea0-0aa0d5074b96'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.764645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13324525110c404ca7d9bab534f5e7ca', 'request_id': 'req-243ac94a-aa30-440e-bea0-0aa0d5074b96'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.779762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] The token cache is being invalidate because project 13324525110c404ca7d9bab534f5e7ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:47.780080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.780772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-243ac94a-aa30-440e-bea0-0aa0d5074b96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.782428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 971/1929] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:47 2022] DELETE /identity/v3/projects/13324525110c404ca7d9bab534f5e7ca => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:47.788431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19340e98-5daa-4ca1-b7d4-847c6a188976 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:47.788728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19340e98-5daa-4ca1-b7d4-847c6a188976 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:47.789276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19340e98-5daa-4ca1-b7d4-847c6a188976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:47.807356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-19340e98-5daa-4ca1-b7d4-847c6a188976 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:47.808647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-19340e98-5daa-4ca1-b7d4-847c6a188976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:47.867625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 972/1930] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:47.875143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3742f5a-d9c8-4a61-bba9-5e822dcf963b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:47.875433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3742f5a-d9c8-4a61-bba9-5e822dcf963b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:47.875750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3742f5a-d9c8-4a61-bba9-5e822dcf963b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:47.893223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a3742f5a-d9c8-4a61-bba9-5e822dcf963b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:47.894553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a3742f5a-d9c8-4a61-bba9-5e822dcf963b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:47.903576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 973/1931] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:47.918963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] The token cache is being invalidate because project 28633dba4a3d4102ae7539ab309525c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:47.918963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.919105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:47.921404 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 959/1932] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:47 2022] DELETE /identity/v3/projects/28633dba4a3d4102ae7539ab309525c3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:47.922099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3742f5a-d9c8-4a61-bba9-5e822dcf963b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:47.925162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:47.930474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:47.936016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3742f5a-d9c8-4a61-bba9-5e822dcf963b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:47.944703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:47.946216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:47.946706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:47.947378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:47.948344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:47.949020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:47.950871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:47.970433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 974/1933] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:47.971605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:47.973436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:47.989799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0f294-cd30-4ef7-bff0-ecb812fdf31b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:48.000509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:48.003421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73e74d-6e15-4535-8ae2-25f4d93d6170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:48.005268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73e74d-6e15-4535-8ae2-25f4d93d6170 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:48.005268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73e74d-6e15-4535-8ae2-25f4d93d6170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:48.005268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73e74d-6e15-4535-8ae2-25f4d93d6170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:48.006489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73e74d-6e15-4535-8ae2-25f4d93d6170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:48.008593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73e74d-6e15-4535-8ae2-25f4d93d6170 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:48.039400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cd88c-92f4-4494-b3d2-4b7b6992b153 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:48.045972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:48.047000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:48.047350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:48.047741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] PATH_INFO: `/v3/users/48ad6d7429f24eff9000d01a5cc3ae98` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:48.049416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ad6d7429f24eff9000d01a5cc3ae98)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:48.051695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:48.184835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 975/1934] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:47 2022] POST /identity/v3/projects => generated 377 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:48.185862 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73e74d-6e15-4535-8ae2-25f4d93d6170 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:48.255631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73e74d-6e15-4535-8ae2-25f4d93d6170 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:48.260923 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7b224-1a33-471d-aa62-84a19230785a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:48.261596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7b224-1a33-471d-aa62-84a19230785a admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:48.261880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7b224-1a33-471d-aa62-84a19230785a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:48.262156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7b224-1a33-471d-aa62-84a19230785a admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:48.262873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7b224-1a33-471d-aa62-84a19230785a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:48.264827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7b224-1a33-471d-aa62-84a19230785a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:48.358693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ad6d7429f24eff9000d01a5cc3ae98'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:48.359445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ad6d7429f24eff9000d01a5cc3ae98'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:48.374493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ad6d7429f24eff9000d01a5cc3ae98'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:48.375390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ad6d7429f24eff9000d01a5cc3ae98'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:48.400363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5ecfbc01-1293-4b8e-9697-9416622a3c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ad6d7429f24eff9000d01a5cc3ae98'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:48.402419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 976/1935] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:48 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:48.406412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7b224-1a33-471d-aa62-84a19230785a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:48.408373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a9b7b224-1a33-471d-aa62-84a19230785a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:48.423510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:48.447761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7b224-1a33-471d-aa62-84a19230785a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:48.453336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123cbdc-a8be-4071-aa4d-db292f997052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:48.454161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123cbdc-a8be-4071-aa4d-db292f997052 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:48.454681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123cbdc-a8be-4071-aa4d-db292f997052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:48.455087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123cbdc-a8be-4071-aa4d-db292f997052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:48.455972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123cbdc-a8be-4071-aa4d-db292f997052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:48.457755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123cbdc-a8be-4071-aa4d-db292f997052 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:48.470494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 977/1936] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:48.474266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123cbdc-a8be-4071-aa4d-db292f997052 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:48.497213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123cbdc-a8be-4071-aa4d-db292f997052 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:48.502400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f2738-be15-4b13-9c50-e344453e3cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:48.503086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f2738-be15-4b13-9c50-e344453e3cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:48.503371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f2738-be15-4b13-9c50-e344453e3cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:48.503701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f2738-be15-4b13-9c50-e344453e3cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:48.504628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f2738-be15-4b13-9c50-e344453e3cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:48.508735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f2738-be15-4b13-9c50-e344453e3cc1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:48.517954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 978/1937] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:48.522046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f2738-be15-4b13-9c50-e344453e3cc1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:48.535988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 960/1938] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:47 2022] DELETE /identity/v3/users/48ad6d7429f24eff9000d01a5cc3ae98 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:48.542630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f2738-be15-4b13-9c50-e344453e3cc1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:48.606734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c12e4-b7f7-4be6-8bc3-0309dab65e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:48.607670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c12e4-b7f7-4be6-8bc3-0309dab65e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:48.607954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c12e4-b7f7-4be6-8bc3-0309dab65e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:48.608230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c12e4-b7f7-4be6-8bc3-0309dab65e24 admin admin] PATH_INFO: `/v3/projects/467854d7e6a4465baea90f2e2b41ca5b/users/2338f10adaad475d9bd735547c593494/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:48.624188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c12e4-b7f7-4be6-8bc3-0309dab65e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=467854d7e6a4465baea90f2e2b41ca5b, user_id=2338f10adaad475d9bd735547c593494, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:48.626916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c12e4-b7f7-4be6-8bc3-0309dab65e24 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:48.702534 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 979/1939] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:48 2022] PUT /identity/v3/projects/467854d7e6a4465baea90f2e2b41ca5b/users/2338f10adaad475d9bd735547c593494/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:48.714731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa45aba-e0d2-441d-a607-492849383cc9 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:48.715027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa45aba-e0d2-441d-a607-492849383cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:48.715310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa45aba-e0d2-441d-a607-492849383cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:48.745643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-5fa45aba-e0d2-441d-a607-492849383cc9 None None] MFA Rules not processed for user `2338f10adaad475d9bd735547c593494`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:48.747145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5fa45aba-e0d2-441d-a607-492849383cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:48.845427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 961/1940] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:48.849707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c12e4-b7f7-4be6-8bc3-0309dab65e24 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:48.937539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c12e4-b7f7-4be6-8bc3-0309dab65e24 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:48.942240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92893d53-e371-4fa9-afaf-ab00326960cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:48.943083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92893d53-e371-4fa9-afaf-ab00326960cb admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:48.943218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92893d53-e371-4fa9-afaf-ab00326960cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:48.944105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92893d53-e371-4fa9-afaf-ab00326960cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:48.944913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92893d53-e371-4fa9-afaf-ab00326960cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:48.946853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92893d53-e371-4fa9-afaf-ab00326960cb admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.063074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 980/1941] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:48 2022] POST /identity/v3/projects => generated 375 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:49.068032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa45aba-e0d2-441d-a607-492849383cc9 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.079060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa45aba-e0d2-441d-a607-492849383cc9 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:49.093952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92893d53-e371-4fa9-afaf-ab00326960cb admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.095697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-92893d53-e371-4fa9-afaf-ab00326960cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:49.142293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76296748-1ed3-430b-995e-6f5de58afb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:49.142967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296748-1ed3-430b-995e-6f5de58afb9b admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.143276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296748-1ed3-430b-995e-6f5de58afb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.143684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296748-1ed3-430b-995e-6f5de58afb9b admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.144523 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76296748-1ed3-430b-995e-6f5de58afb9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:49.146441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76296748-1ed3-430b-995e-6f5de58afb9b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.169355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92893d53-e371-4fa9-afaf-ab00326960cb admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:49.174148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:49.175010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.175881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.176164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] PATH_INFO: `/v3/projects/5d0f00eeb4924a40a6fb7acdef0caad0` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.177362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d0f00eeb4924a40a6fb7acdef0caad0)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:49.179810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.285490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 962/1942] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:49 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:49.286807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76296748-1ed3-430b-995e-6f5de58afb9b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.297388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76296748-1ed3-430b-995e-6f5de58afb9b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:49.303726 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684f89-7a80-4299-9733-4e94add6b6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:49.304416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684f89-7a80-4299-9733-4e94add6b6de admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.304678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684f89-7a80-4299-9733-4e94add6b6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.305198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684f89-7a80-4299-9733-4e94add6b6de admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.305939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38684f89-7a80-4299-9733-4e94add6b6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:49.307552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38684f89-7a80-4299-9733-4e94add6b6de admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.315929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d0f00eeb4924a40a6fb7acdef0caad0', 'request_id': 'req-45a669b2-d227-445c-bae3-b0f0c00a1eea'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:49.322203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 963/1943] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:49.323519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684f89-7a80-4299-9733-4e94add6b6de admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.334159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684f89-7a80-4299-9733-4e94add6b6de admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:49.404494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c255cc-ae23-4b56-8fac-e69605cb27d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:49.405498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c255cc-ae23-4b56-8fac-e69605cb27d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.405672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c255cc-ae23-4b56-8fac-e69605cb27d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.405951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c255cc-ae23-4b56-8fac-e69605cb27d7 admin admin] PATH_INFO: `/v3/projects/4bfe924bf9c24328b9a3c346769fb942/users/f91ef3807664477fafe7f40c7412e62a/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.410524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d0f00eeb4924a40a6fb7acdef0caad0', 'request_id': 'req-45a669b2-d227-445c-bae3-b0f0c00a1eea'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:49.418057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d0f00eeb4924a40a6fb7acdef0caad0', 'request_id': 'req-45a669b2-d227-445c-bae3-b0f0c00a1eea'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:49.424168 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c255cc-ae23-4b56-8fac-e69605cb27d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bfe924bf9c24328b9a3c346769fb942, user_id=f91ef3807664477fafe7f40c7412e62a, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:49.427699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c255cc-ae23-4b56-8fac-e69605cb27d7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.432064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] The token cache is being invalidate because project 5d0f00eeb4924a40a6fb7acdef0caad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:49.432189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:49.432890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-45a669b2-d227-445c-bae3-b0f0c00a1eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:49.434882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 981/1944] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:49 2022] DELETE /identity/v3/projects/5d0f00eeb4924a40a6fb7acdef0caad0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:49.443143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce2582-6b3d-470f-a935-09e67e136a92 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.443143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce2582-6b3d-470f-a935-09e67e136a92 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.443143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce2582-6b3d-470f-a935-09e67e136a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.464611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-06ce2582-6b3d-470f-a935-09e67e136a92 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:49.465819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-06ce2582-6b3d-470f-a935-09e67e136a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:49.474251 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 982/1945] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:49.482484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f73b8-08bb-48a8-b7c7-53b7c2b17739 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.482782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f73b8-08bb-48a8-b7c7-53b7c2b17739 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.483036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f73b8-08bb-48a8-b7c7-53b7c2b17739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.504876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a22f73b8-08bb-48a8-b7c7-53b7c2b17739 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:49.506679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a22f73b8-08bb-48a8-b7c7-53b7c2b17739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:49.515937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 983/1946] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:49.520219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f73b8-08bb-48a8-b7c7-53b7c2b17739 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.533738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f73b8-08bb-48a8-b7c7-53b7c2b17739 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:49.539353 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bf050-656c-4cca-b4b5-88f892893c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:49.540051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bf050-656c-4cca-b4b5-88f892893c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.540428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bf050-656c-4cca-b4b5-88f892893c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.540678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bf050-656c-4cca-b4b5-88f892893c8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.541471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bf050-656c-4cca-b4b5-88f892893c8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:49.541773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bf050-656c-4cca-b4b5-88f892893c8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:49.543338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bf050-656c-4cca-b4b5-88f892893c8b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.556803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 984/1947] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:49.557731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 964/1948] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:49 2022] PUT /identity/v3/projects/4bfe924bf9c24328b9a3c346769fb942/users/f91ef3807664477fafe7f40c7412e62a/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:49.562879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bf050-656c-4cca-b4b5-88f892893c8b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.563030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c255cc-ae23-4b56-8fac-e69605cb27d7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.564414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-19c255cc-ae23-4b56-8fac-e69605cb27d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:49.581022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bf050-656c-4cca-b4b5-88f892893c8b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:49.643233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c3a62-9aef-4e9d-b356-4297e97ea5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:49.644143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c3a62-9aef-4e9d-b356-4297e97ea5ee admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.649469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c3a62-9aef-4e9d-b356-4297e97ea5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.649469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c3a62-9aef-4e9d-b356-4297e97ea5ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.649469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c3a62-9aef-4e9d-b356-4297e97ea5ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:49.649469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c3a62-9aef-4e9d-b356-4297e97ea5ee admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.691479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c255cc-ae23-4b56-8fac-e69605cb27d7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:49.696418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129a29f-98e2-4324-b60b-3f2a2ac9254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:49.697091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129a29f-98e2-4324-b60b-3f2a2ac9254e admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.697369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129a29f-98e2-4324-b60b-3f2a2ac9254e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.697643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129a29f-98e2-4324-b60b-3f2a2ac9254e admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.698368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129a29f-98e2-4324-b60b-3f2a2ac9254e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:49.700156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129a29f-98e2-4324-b60b-3f2a2ac9254e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.710543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 965/1949] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:49.716797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129a29f-98e2-4324-b60b-3f2a2ac9254e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.729897 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129a29f-98e2-4324-b60b-3f2a2ac9254e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:49.735270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e151-367c-4641-a92f-40be8baf544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:49.736429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e151-367c-4641-a92f-40be8baf544e admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.736861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e151-367c-4641-a92f-40be8baf544e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.737139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e151-367c-4641-a92f-40be8baf544e admin admin] PATH_INFO: `/v3/projects/4bfe924bf9c24328b9a3c346769fb942/users/f91ef3807664477fafe7f40c7412e62a/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.739444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287e151-367c-4641-a92f-40be8baf544e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bfe924bf9c24328b9a3c346769fb942, user_id=f91ef3807664477fafe7f40c7412e62a, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:49.742865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287e151-367c-4641-a92f-40be8baf544e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.755083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 966/1950] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:49 2022] PUT /identity/v3/projects/4bfe924bf9c24328b9a3c346769fb942/users/f91ef3807664477fafe7f40c7412e62a/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:49.767272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:49.769878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b7d3c-50ae-4e72-b86e-733b7e5b1f15 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.770165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b7d3c-50ae-4e72-b86e-733b7e5b1f15 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.770441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b7d3c-50ae-4e72-b86e-733b7e5b1f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.782525 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 985/1951] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:49 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:49.783835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c3a62-9aef-4e9d-b356-4297e97ea5ee admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.810491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8c0b7d3c-50ae-4e72-b86e-733b7e5b1f15 None None] MFA Rules not processed for user `f91ef3807664477fafe7f40c7412e62a`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:49.815417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8c0b7d3c-50ae-4e72-b86e-733b7e5b1f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:49.896412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c3a62-9aef-4e9d-b356-4297e97ea5ee admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:49.903244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9727afb-f016-4aa5-881a-3d6fb1640e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:49.903973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9727afb-f016-4aa5-881a-3d6fb1640e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:49.904260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9727afb-f016-4aa5-881a-3d6fb1640e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:49.904541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9727afb-f016-4aa5-881a-3d6fb1640e6a admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:49.905242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9727afb-f016-4aa5-881a-3d6fb1640e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:49.907579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9727afb-f016-4aa5-881a-3d6fb1640e6a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:49.910034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:49.916047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 967/1952] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:49.927281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b7d3c-50ae-4e72-b86e-733b7e5b1f15 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:49.929326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8c0b7d3c-50ae-4e72-b86e-733b7e5b1f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:49.995844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b7d3c-50ae-4e72-b86e-733b7e5b1f15 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:50.002234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4786d17-d2ca-49a9-b9bd-58b389ac4a5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:50.003502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4786d17-d2ca-49a9-b9bd-58b389ac4a5b service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.004299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4786d17-d2ca-49a9-b9bd-58b389ac4a5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.004583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4786d17-d2ca-49a9-b9bd-58b389ac4a5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.006930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c4786d17-d2ca-49a9-b9bd-58b389ac4a5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:50.022431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 986/1953] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:49 2022] POST /identity/v3/users => generated 294 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:50.023551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9727afb-f016-4aa5-881a-3d6fb1640e6a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:50.042691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9727afb-f016-4aa5-881a-3d6fb1640e6a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:50.051986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cb04f-c947-43a2-ab39-b0a0db5beec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:50.052719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cb04f-c947-43a2-ab39-b0a0db5beec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.059270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cb04f-c947-43a2-ab39-b0a0db5beec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.059836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cb04f-c947-43a2-ab39-b0a0db5beec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.060928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cb04f-c947-43a2-ab39-b0a0db5beec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:50.062567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cb04f-c947-43a2-ab39-b0a0db5beec5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:50.078696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 987/1954] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:50.082639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cb04f-c947-43a2-ab39-b0a0db5beec5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:50.091763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4786d17-d2ca-49a9-b9bd-58b389ac4a5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:50.093174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4786d17-d2ca-49a9-b9bd-58b389ac4a5b service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:50.097148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cb04f-c947-43a2-ab39-b0a0db5beec5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:50.108608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 968/1955] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:36:50.108985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b6007-1017-4d99-b416-2c85d7a9049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:50.109783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b6007-1017-4d99-b416-2c85d7a9049c admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.110231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b6007-1017-4d99-b416-2c85d7a9049c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.110976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b6007-1017-4d99-b416-2c85d7a9049c admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.112107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b6007-1017-4d99-b416-2c85d7a9049c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:50.114177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b6007-1017-4d99-b416-2c85d7a9049c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:50.127027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 988/1956] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:50.130825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4786d17-d2ca-49a9-b9bd-58b389ac4a5b service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:50.141936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4786d17-d2ca-49a9-b9bd-58b389ac4a5b service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:50.147684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5ee4e-c0d1-4c59-81c5-79c25b89b89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:50.148769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5ee4e-c0d1-4c59-81c5-79c25b89b89c admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.149066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5ee4e-c0d1-4c59-81c5-79c25b89b89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.149427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5ee4e-c0d1-4c59-81c5-79c25b89b89c admin admin] PATH_INFO: `/v3/projects/5813719fb11e4a3692f984f58149915f/users/81481758775445e3a5c69d503f3293f7/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.171700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5ee4e-c0d1-4c59-81c5-79c25b89b89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5813719fb11e4a3692f984f58149915f, user_id=81481758775445e3a5c69d503f3293f7, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:50.175776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5ee4e-c0d1-4c59-81c5-79c25b89b89c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:50.228928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa5859-649e-4fac-b78b-22cf43b27a65 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.229825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa5859-649e-4fac-b78b-22cf43b27a65 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.229825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa5859-649e-4fac-b78b-22cf43b27a65 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.231935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 989/1957] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:36:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:36:50.238793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184dce2-2b42-4805-9931-027b7e204f81 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.240446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184dce2-2b42-4805-9931-027b7e204f81 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.240810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184dce2-2b42-4805-9931-027b7e204f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.263706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-b184dce2-2b42-4805-9931-027b7e204f81 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:50.265058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b184dce2-2b42-4805-9931-027b7e204f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:50.302315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 969/1958] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:50 2022] PUT /identity/v3/projects/5813719fb11e4a3692f984f58149915f/users/81481758775445e3a5c69d503f3293f7/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:50.313416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6ed49-a64b-43ae-a1b0-6f8ff1cbe8d4 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.313717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6ed49-a64b-43ae-a1b0-6f8ff1cbe8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.314003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6ed49-a64b-43ae-a1b0-6f8ff1cbe8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.345760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4bf6ed49-a64b-43ae-a1b0-6f8ff1cbe8d4 None None] MFA Rules not processed for user `81481758775445e3a5c69d503f3293f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:50.347101 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4bf6ed49-a64b-43ae-a1b0-6f8ff1cbe8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:50.356773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 990/1959] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:36:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:50.375778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184dce2-2b42-4805-9931-027b7e204f81 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:50.376879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b184dce2-2b42-4805-9931-027b7e204f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:50.392223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184dce2-2b42-4805-9931-027b7e204f81 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:50.444464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 970/1960] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:50.447287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6ed49-a64b-43ae-a1b0-6f8ff1cbe8d4 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:50.475521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ef85a-c53f-452e-978d-dcb784ef30ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:50.476247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ef85a-c53f-452e-978d-dcb784ef30ea service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.476526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ef85a-c53f-452e-978d-dcb784ef30ea service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.476800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ef85a-c53f-452e-978d-dcb784ef30ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.487916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ef85a-c53f-452e-978d-dcb784ef30ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:50.489953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ef85a-c53f-452e-978d-dcb784ef30ea service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:50.506608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 991/1961] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:50.514574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6ed49-a64b-43ae-a1b0-6f8ff1cbe8d4 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:50.526591 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966edd-b8b3-40bb-a693-7bc39f9132c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:50.527315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966edd-b8b3-40bb-a693-7bc39f9132c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.527921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966edd-b8b3-40bb-a693-7bc39f9132c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.528209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966edd-b8b3-40bb-a693-7bc39f9132c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.528925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0966edd-b8b3-40bb-a693-7bc39f9132c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:50.531280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0966edd-b8b3-40bb-a693-7bc39f9132c1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:50.626360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 971/1962] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:50 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:50.631474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ef85a-c53f-452e-978d-dcb784ef30ea service nova] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:50.643606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ef85a-c53f-452e-978d-dcb784ef30ea service nova] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:50.701275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0425c24-aa74-46f4-b86a-c9e1b097f0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:50.701993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0425c24-aa74-46f4-b86a-c9e1b097f0dc admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.702284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0425c24-aa74-46f4-b86a-c9e1b097f0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.702562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0425c24-aa74-46f4-b86a-c9e1b097f0dc admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.703273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0425c24-aa74-46f4-b86a-c9e1b097f0dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:50.705321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0425c24-aa74-46f4-b86a-c9e1b097f0dc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:50.795357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 992/1963] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:50 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:50.800016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966edd-b8b3-40bb-a693-7bc39f9132c1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:50.815565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966edd-b8b3-40bb-a693-7bc39f9132c1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:50.822763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e042dd8-dbee-4e48-b995-a7d895854a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:50.823602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e042dd8-dbee-4e48-b995-a7d895854a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.824504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e042dd8-dbee-4e48-b995-a7d895854a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.824504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e042dd8-dbee-4e48-b995-a7d895854a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.825422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e042dd8-dbee-4e48-b995-a7d895854a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:50.827211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e042dd8-dbee-4e48-b995-a7d895854a0e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:50.853090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 972/1964] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:50.856462 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0425c24-aa74-46f4-b86a-c9e1b097f0dc admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:50.877746 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0425c24-aa74-46f4-b86a-c9e1b097f0dc admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:50.890517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01750d12-223a-459c-9a91-13aa54567be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:50.892130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01750d12-223a-459c-9a91-13aa54567be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:50.892486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01750d12-223a-459c-9a91-13aa54567be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:50.892842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01750d12-223a-459c-9a91-13aa54567be5 admin admin] PATH_INFO: `/v3/projects/08d3e198b38f4d3a92c39bb9142cf53f/users/1ae26215095d4eebafbb30514835f494/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:50.911828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01750d12-223a-459c-9a91-13aa54567be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d3e198b38f4d3a92c39bb9142cf53f, user_id=1ae26215095d4eebafbb30514835f494, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:50.923060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01750d12-223a-459c-9a91-13aa54567be5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:51.034245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 993/1965] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:50 2022] PUT /identity/v3/projects/08d3e198b38f4d3a92c39bb9142cf53f/users/1ae26215095d4eebafbb30514835f494/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:51.042486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e042dd8-dbee-4e48-b995-a7d895854a0e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:51.062909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e042dd8-dbee-4e48-b995-a7d895854a0e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:51.114537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8021e10-c62c-498a-8cea-12a6b664f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:51.115129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8021e10-c62c-498a-8cea-12a6b664f5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:51.116024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8021e10-c62c-498a-8cea-12a6b664f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:51.116303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8021e10-c62c-498a-8cea-12a6b664f5a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:51.117032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8021e10-c62c-498a-8cea-12a6b664f5a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:51.118877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8021e10-c62c-498a-8cea-12a6b664f5a6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:51.131756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 973/1966] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:51.133037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01750d12-223a-459c-9a91-13aa54567be5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:51.143334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01750d12-223a-459c-9a91-13aa54567be5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:51.148690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc827b8-172d-45e7-a5aa-9c8deb438bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:51.149587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc827b8-172d-45e7-a5aa-9c8deb438bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:51.149944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc827b8-172d-45e7-a5aa-9c8deb438bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:51.150209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc827b8-172d-45e7-a5aa-9c8deb438bd7 admin admin] PATH_INFO: `/v3/projects/08d3e198b38f4d3a92c39bb9142cf53f/users/1ae26215095d4eebafbb30514835f494/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:51.152246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc827b8-172d-45e7-a5aa-9c8deb438bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d3e198b38f4d3a92c39bb9142cf53f, user_id=1ae26215095d4eebafbb30514835f494, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:51.155333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc827b8-172d-45e7-a5aa-9c8deb438bd7 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:51.180909 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 994/1967] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:51 2022] PUT /identity/v3/projects/08d3e198b38f4d3a92c39bb9142cf53f/users/1ae26215095d4eebafbb30514835f494/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:51.191885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec0eb7-9f41-49ca-a81d-3f91c7ea43eb None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:51.192183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec0eb7-9f41-49ca-a81d-3f91c7ea43eb None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:51.192466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec0eb7-9f41-49ca-a81d-3f91c7ea43eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:51.226293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-edec0eb7-9f41-49ca-a81d-3f91c7ea43eb None None] MFA Rules not processed for user `1ae26215095d4eebafbb30514835f494`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:51.227572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-edec0eb7-9f41-49ca-a81d-3f91c7ea43eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:51.250693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:51.328883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 974/1968] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:51.350263 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc827b8-172d-45e7-a5aa-9c8deb438bd7 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:51.352784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fdc827b8-172d-45e7-a5aa-9c8deb438bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:51.420996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc827b8-172d-45e7-a5aa-9c8deb438bd7 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:51.430012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb438b-2d4b-4991-a9e6-8448cc6ca0fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:51.430678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb438b-2d4b-4991-a9e6-8448cc6ca0fc service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:51.430955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb438b-2d4b-4991-a9e6-8448cc6ca0fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:51.431229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb438b-2d4b-4991-a9e6-8448cc6ca0fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:51.441550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb438b-2d4b-4991-a9e6-8448cc6ca0fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:51.443032 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb438b-2d4b-4991-a9e6-8448cc6ca0fc service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:51.452788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 995/1969] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:51.514312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:51.522948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec0eb7-9f41-49ca-a81d-3f91c7ea43eb None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:51.590098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec0eb7-9f41-49ca-a81d-3f91c7ea43eb None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:51.594668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:51.595575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:51.595875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:51.596151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] PATH_INFO: `/v3/users/81481758775445e3a5c69d503f3293f7` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:51.597496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=81481758775445e3a5c69d503f3293f7)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:51.599448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:51.790593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81481758775445e3a5c69d503f3293f7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:51.791255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81481758775445e3a5c69d503f3293f7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:51.798127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81481758775445e3a5c69d503f3293f7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:51.798804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81481758775445e3a5c69d503f3293f7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:51.810759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81481758775445e3a5c69d503f3293f7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:51.885780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 975/1970] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:51 2022] DELETE /identity/v3/users/81481758775445e3a5c69d503f3293f7 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:51.891043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0246d6-4ea7-45ac-95e2-8e37a47ba91a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:51.891338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0246d6-4ea7-45ac-95e2-8e37a47ba91a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:51.891654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0246d6-4ea7-45ac-95e2-8e37a47ba91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:51.912017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7c0246d6-4ea7-45ac-95e2-8e37a47ba91a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:51.913470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7c0246d6-4ea7-45ac-95e2-8e37a47ba91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:51.990823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 996/1971] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:52.001618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:52.003028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:52.063743 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0246d6-4ea7-45ac-95e2-8e37a47ba91a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:52.078175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7c0246d6-4ea7-45ac-95e2-8e37a47ba91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:52.079032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15183020-31ab-4cd6-8119-f7afb95248aa admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:52.082222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0246d6-4ea7-45ac-95e2-8e37a47ba91a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:52.084860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27b43c-eb99-45c2-a130-d9c33d4a20ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:52.085532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27b43c-eb99-45c2-a130-d9c33d4a20ed service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:52.085903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27b43c-eb99-45c2-a130-d9c33d4a20ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:52.086306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27b43c-eb99-45c2-a130-d9c33d4a20ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:52.089358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:52.089358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:52.095504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:52.095504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] PATH_INFO: `/v3/users/ecd3ddc472d54a0b981707a3497c0255` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:52.095504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd3ddc472d54a0b981707a3497c0255)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:52.095504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:52.098338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27b43c-eb99-45c2-a130-d9c33d4a20ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:52.100020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27b43c-eb99-45c2-a130-d9c33d4a20ed service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:52.132317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 976/1972] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:52.532540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd3ddc472d54a0b981707a3497c0255'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:52.592024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd3ddc472d54a0b981707a3497c0255'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:52.595632 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd3ddc472d54a0b981707a3497c0255'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:52.607938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd3ddc472d54a0b981707a3497c0255'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:52.609086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd3ddc472d54a0b981707a3497c0255'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:52.645354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 997/1973] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:52 2022] DELETE /identity/v3/users/ecd3ddc472d54a0b981707a3497c0255 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:52.647163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fa9cc-69b7-4dda-80ed-c5ff95f54b3a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:52.647640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fa9cc-69b7-4dda-80ed-c5ff95f54b3a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:52.647765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fa9cc-69b7-4dda-80ed-c5ff95f54b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:52.668399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-757fa9cc-69b7-4dda-80ed-c5ff95f54b3a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:52.669938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-757fa9cc-69b7-4dda-80ed-c5ff95f54b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:52.733782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 977/1974] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:52.744372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:52.746233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:52.811558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cca9b4-f11a-49b6-a3a2-b5b6945122c1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:52.812526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fa9cc-69b7-4dda-80ed-c5ff95f54b3a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:52.814039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-757fa9cc-69b7-4dda-80ed-c5ff95f54b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:52.816700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaba020-3c85-43d6-9c2a-ed2b3937f167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:52.817354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaba020-3c85-43d6-9c2a-ed2b3937f167 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:52.817635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaba020-3c85-43d6-9c2a-ed2b3937f167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:52.817904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaba020-3c85-43d6-9c2a-ed2b3937f167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:52.828163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaba020-3c85-43d6-9c2a-ed2b3937f167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:52.829290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fa9cc-69b7-4dda-80ed-c5ff95f54b3a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:52.829603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaba020-3c85-43d6-9c2a-ed2b3937f167 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:52.837312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:52.838822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:52.839277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:52.839799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] PATH_INFO: `/v3/projects/5813719fb11e4a3692f984f58149915f` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:52.841243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5813719fb11e4a3692f984f58149915f)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:52.842177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 998/1975] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:52.848358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:52.976331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5813719fb11e4a3692f984f58149915f', 'request_id': 'req-d645da6b-70c3-436a-99d9-7e64517c2081'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:52.983664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5813719fb11e4a3692f984f58149915f', 'request_id': 'req-d645da6b-70c3-436a-99d9-7e64517c2081'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:52.990963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5813719fb11e4a3692f984f58149915f', 'request_id': 'req-d645da6b-70c3-436a-99d9-7e64517c2081'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.095995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] The token cache is being invalidate because project 5813719fb11e4a3692f984f58149915f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:53.096377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.097135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.099078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 978/1976] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:52 2022] DELETE /identity/v3/projects/5813719fb11e4a3692f984f58149915f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:53.103730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaba020-3c85-43d6-9c2a-ed2b3937f167 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:53.118575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaba020-3c85-43d6-9c2a-ed2b3937f167 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:53.180348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:53.181200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:53.181482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:53.181756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] PATH_INFO: `/v3/users/1ae26215095d4eebafbb30514835f494` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:53.182954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ae26215095d4eebafbb30514835f494)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:53.184950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:53.354648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:53.362116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:53.363608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:53.378036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645da6b-70c3-436a-99d9-7e64517c2081 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:53.390092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:53.392806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:53.393249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:53.393669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] PATH_INFO: `/v3/projects/5194ccc9cb9e4a79820a1fa134b32726` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:53.396040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5194ccc9cb9e4a79820a1fa134b32726)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:53.398717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:53.427497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ae26215095d4eebafbb30514835f494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.497439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ae26215095d4eebafbb30514835f494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.498436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ae26215095d4eebafbb30514835f494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.499392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:53.507685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ae26215095d4eebafbb30514835f494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.508343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dd74f402-615d-402c-b566-3984377a1ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ae26215095d4eebafbb30514835f494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.519769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5194ccc9cb9e4a79820a1fa134b32726', 'request_id': 'req-462a9064-afa5-4e8f-8eb8-c5cb8046e508'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.529220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5194ccc9cb9e4a79820a1fa134b32726', 'request_id': 'req-462a9064-afa5-4e8f-8eb8-c5cb8046e508'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.529338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 999/1977] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:53 2022] DELETE /identity/v3/users/1ae26215095d4eebafbb30514835f494 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:53.534483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5194ccc9cb9e4a79820a1fa134b32726', 'request_id': 'req-462a9064-afa5-4e8f-8eb8-c5cb8046e508'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.593014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] The token cache is being invalidate because project 5194ccc9cb9e4a79820a1fa134b32726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:53.593444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.594195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-462a9064-afa5-4e8f-8eb8-c5cb8046e508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:53.596378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 979/1978] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:53 2022] DELETE /identity/v3/projects/5194ccc9cb9e4a79820a1fa134b32726 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:53.601395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15fabe-63be-4bb8-b423-bf06208068b1 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:53.601696 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15fabe-63be-4bb8-b423-bf06208068b1 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:53.601978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15fabe-63be-4bb8-b423-bf06208068b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:53.625639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-de15fabe-63be-4bb8-b423-bf06208068b1 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:53.626907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-de15fabe-63be-4bb8-b423-bf06208068b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:53.689277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1000/1979] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:53.697496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231ed9c-0507-4bff-8837-8c39d0cdf578 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:53.698021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231ed9c-0507-4bff-8837-8c39d0cdf578 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:53.698207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231ed9c-0507-4bff-8837-8c39d0cdf578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:53.716807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-6231ed9c-0507-4bff-8837-8c39d0cdf578 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:53.718641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6231ed9c-0507-4bff-8837-8c39d0cdf578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:53.726928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 980/1980] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:53.731105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15fabe-63be-4bb8-b423-bf06208068b1 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:53.742952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15fabe-63be-4bb8-b423-bf06208068b1 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:53.758159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283c897-4952-483f-9ec2-ac0b60138ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:53.762381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283c897-4952-483f-9ec2-ac0b60138ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:53.762381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283c897-4952-483f-9ec2-ac0b60138ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:53.762381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283c897-4952-483f-9ec2-ac0b60138ae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:53.762381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283c897-4952-483f-9ec2-ac0b60138ae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:53.762381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283c897-4952-483f-9ec2-ac0b60138ae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:53.762381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283c897-4952-483f-9ec2-ac0b60138ae5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:53.781722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1001/1981] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:36:53.785214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231ed9c-0507-4bff-8837-8c39d0cdf578 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:53.793818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231ed9c-0507-4bff-8837-8c39d0cdf578 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:53.798498 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f379549-f99c-4cec-aac5-be226d1dc68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:53.799150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f379549-f99c-4cec-aac5-be226d1dc68a admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:53.799429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f379549-f99c-4cec-aac5-be226d1dc68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:53.799746 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f379549-f99c-4cec-aac5-be226d1dc68a admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:53.800433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f379549-f99c-4cec-aac5-be226d1dc68a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:53.802312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f379549-f99c-4cec-aac5-be226d1dc68a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:53.881713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 981/1982] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:53 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:53.885890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283c897-4952-483f-9ec2-ac0b60138ae5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:53.896836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283c897-4952-483f-9ec2-ac0b60138ae5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:53.954878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb23138-7327-44e1-82dc-fec4b44298d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:53.955489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb23138-7327-44e1-82dc-fec4b44298d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:53.955802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb23138-7327-44e1-82dc-fec4b44298d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:53.956080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb23138-7327-44e1-82dc-fec4b44298d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:53.956779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb23138-7327-44e1-82dc-fec4b44298d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:53.958706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb23138-7327-44e1-82dc-fec4b44298d6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:53.983440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f379549-f99c-4cec-aac5-be226d1dc68a admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:53.985349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5f379549-f99c-4cec-aac5-be226d1dc68a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:54.001938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f379549-f99c-4cec-aac5-be226d1dc68a admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.008688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.009665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.009945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.010220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] PATH_INFO: `/v3/projects/08d3e198b38f4d3a92c39bb9142cf53f` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.011478 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d3e198b38f4d3a92c39bb9142cf53f)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.013811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.067240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1002/1983] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:53 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:54.068497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb23138-7327-44e1-82dc-fec4b44298d6 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.078664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb23138-7327-44e1-82dc-fec4b44298d6 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.083329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ce894-d722-472b-982c-9c1180c57c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.084243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ce894-d722-472b-982c-9c1180c57c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.084766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ce894-d722-472b-982c-9c1180c57c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.085074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ce894-d722-472b-982c-9c1180c57c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.090233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:54.090233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ce894-d722-472b-982c-9c1180c57c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.090233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ce894-d722-472b-982c-9c1180c57c47 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.100789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1003/1984] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:54.104684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ce894-d722-472b-982c-9c1180c57c47 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.114910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ce894-d722-472b-982c-9c1180c57c47 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.120561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2553e-1400-4cb8-b0ef-e05accb5de17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.121221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2553e-1400-4cb8-b0ef-e05accb5de17 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.121504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2553e-1400-4cb8-b0ef-e05accb5de17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.121783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2553e-1400-4cb8-b0ef-e05accb5de17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.122515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2553e-1400-4cb8-b0ef-e05accb5de17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.124269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2553e-1400-4cb8-b0ef-e05accb5de17 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.135662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1004/1985] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:54.139732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2553e-1400-4cb8-b0ef-e05accb5de17 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.149381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2553e-1400-4cb8-b0ef-e05accb5de17 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.154132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e18fe1-0584-4cbd-a11b-d48dc5883e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.154985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e18fe1-0584-4cbd-a11b-d48dc5883e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.155267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e18fe1-0584-4cbd-a11b-d48dc5883e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.155559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e18fe1-0584-4cbd-a11b-d48dc5883e8b admin admin] PATH_INFO: `/v3/projects/409c153f006d4120a5fc925a25de409a/users/8f789cfeb2d74ac999de56ed05f59d31/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.171124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e18fe1-0584-4cbd-a11b-d48dc5883e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=409c153f006d4120a5fc925a25de409a, user_id=8f789cfeb2d74ac999de56ed05f59d31, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.173873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e18fe1-0584-4cbd-a11b-d48dc5883e8b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.269517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1005/1986] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:54 2022] PUT /identity/v3/projects/409c153f006d4120a5fc925a25de409a/users/8f789cfeb2d74ac999de56ed05f59d31/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:54.279905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3bde8-55df-4d42-a615-820e178a74cf None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.280044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:54.280468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3bde8-55df-4d42-a615-820e178a74cf None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.280742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d3e198b38f4d3a92c39bb9142cf53f', 'request_id': 'req-84b4543c-d5b1-4554-b398-33fedfe4931b'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:54.281100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3bde8-55df-4d42-a615-820e178a74cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.286722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d3e198b38f4d3a92c39bb9142cf53f', 'request_id': 'req-84b4543c-d5b1-4554-b398-33fedfe4931b'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:54.295931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d3e198b38f4d3a92c39bb9142cf53f', 'request_id': 'req-84b4543c-d5b1-4554-b398-33fedfe4931b'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:54.309923 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5ea3bde8-55df-4d42-a615-820e178a74cf None None] MFA Rules not processed for user `8f789cfeb2d74ac999de56ed05f59d31`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:54.311167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5ea3bde8-55df-4d42-a615-820e178a74cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:54.365750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] The token cache is being invalidate because project 08d3e198b38f4d3a92c39bb9142cf53f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:54.366124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:54.366833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-84b4543c-d5b1-4554-b398-33fedfe4931b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:54.369863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 982/1987] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:53 2022] DELETE /identity/v3/projects/08d3e198b38f4d3a92c39bb9142cf53f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:54.374649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25054b06-746d-4021-ad00-0d1894d9d450 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.374937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25054b06-746d-4021-ad00-0d1894d9d450 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.375213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25054b06-746d-4021-ad00-0d1894d9d450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.378048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1006/1988] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:54.382275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3bde8-55df-4d42-a615-820e178a74cf None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.398021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-25054b06-746d-4021-ad00-0d1894d9d450 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:54.399238 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-25054b06-746d-4021-ad00-0d1894d9d450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:54.446138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3bde8-55df-4d42-a615-820e178a74cf None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.450908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd35e1-2091-4bb1-9fca-66f0746d339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.451606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd35e1-2091-4bb1-9fca-66f0746d339a admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.451891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd35e1-2091-4bb1-9fca-66f0746d339a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.452157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd35e1-2091-4bb1-9fca-66f0746d339a admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.452843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd35e1-2091-4bb1-9fca-66f0746d339a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.459410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd35e1-2091-4bb1-9fca-66f0746d339a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.463351 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 983/1989] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:54.471002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55291057-b7ba-4453-a1b1-de4f80df5938 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.471416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55291057-b7ba-4453-a1b1-de4f80df5938 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.475286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55291057-b7ba-4453-a1b1-de4f80df5938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.493438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-55291057-b7ba-4453-a1b1-de4f80df5938 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:54.494703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-55291057-b7ba-4453-a1b1-de4f80df5938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:54.509526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 984/1990] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:54.512188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55291057-b7ba-4453-a1b1-de4f80df5938 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.526612 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55291057-b7ba-4453-a1b1-de4f80df5938 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.537661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1007/1991] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:54 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:54.537779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94afd72-cb36-4807-af5a-cb8f2eb427f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.537779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94afd72-cb36-4807-af5a-cb8f2eb427f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.537779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94afd72-cb36-4807-af5a-cb8f2eb427f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.537779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94afd72-cb36-4807-af5a-cb8f2eb427f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.539507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94afd72-cb36-4807-af5a-cb8f2eb427f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:54.539507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94afd72-cb36-4807-af5a-cb8f2eb427f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.539507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94afd72-cb36-4807-af5a-cb8f2eb427f5 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.542419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd35e1-2091-4bb1-9fca-66f0746d339a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.544534 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-cbfd35e1-2091-4bb1-9fca-66f0746d339a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:54.559750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 985/1992] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:36:54.564284 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94afd72-cb36-4807-af5a-cb8f2eb427f5 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.575827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94afd72-cb36-4807-af5a-cb8f2eb427f5 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.644579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:54.655456 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd35e1-2091-4bb1-9fca-66f0746d339a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.664101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250110a3-94c7-472c-9cbb-4614871c5e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.665154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250110a3-94c7-472c-9cbb-4614871c5e87 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.665443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250110a3-94c7-472c-9cbb-4614871c5e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.665723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250110a3-94c7-472c-9cbb-4614871c5e87 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.666962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250110a3-94c7-472c-9cbb-4614871c5e87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.668781 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:54.669103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250110a3-94c7-472c-9cbb-4614871c5e87 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.700312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1347a-3963-457c-b70e-02d16bbf88d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.700989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1347a-3963-457c-b70e-02d16bbf88d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.701273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1347a-3963-457c-b70e-02d16bbf88d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.701551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1347a-3963-457c-b70e-02d16bbf88d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.702345 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1347a-3963-457c-b70e-02d16bbf88d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.704320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1347a-3963-457c-b70e-02d16bbf88d7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.760302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1008/1993] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:54 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:54.763874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250110a3-94c7-472c-9cbb-4614871c5e87 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.774504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250110a3-94c7-472c-9cbb-4614871c5e87 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.785703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76539b2-18b3-4e22-9bad-e1bf8b42b923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.785703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76539b2-18b3-4e22-9bad-e1bf8b42b923 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.785703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76539b2-18b3-4e22-9bad-e1bf8b42b923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.785703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76539b2-18b3-4e22-9bad-e1bf8b42b923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.786166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76539b2-18b3-4e22-9bad-e1bf8b42b923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.786166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76539b2-18b3-4e22-9bad-e1bf8b42b923 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.796569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1009/1995] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:54.796902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 986/1995] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:54 2022] POST /identity/v3/projects => generated 381 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:54.797536 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76539b2-18b3-4e22-9bad-e1bf8b42b923 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.800273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1347a-3963-457c-b70e-02d16bbf88d7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.807476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76539b2-18b3-4e22-9bad-e1bf8b42b923 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.869464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1347a-3963-457c-b70e-02d16bbf88d7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.875006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13bcbd-36d1-48d9-947b-ee3f381d64d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.875169 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3603e7-a2ea-449d-a652-801921246fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.875808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3603e7-a2ea-449d-a652-801921246fac admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.875961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13bcbd-36d1-48d9-947b-ee3f381d64d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.876092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3603e7-a2ea-449d-a652-801921246fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.876248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13bcbd-36d1-48d9-947b-ee3f381d64d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.876369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3603e7-a2ea-449d-a652-801921246fac admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.876527 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13bcbd-36d1-48d9-947b-ee3f381d64d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.877142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3603e7-a2ea-449d-a652-801921246fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.877480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13bcbd-36d1-48d9-947b-ee3f381d64d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.879440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13bcbd-36d1-48d9-947b-ee3f381d64d3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.879978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3603e7-a2ea-449d-a652-801921246fac admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:54.892019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 987/1996] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:54.895815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13bcbd-36d1-48d9-947b-ee3f381d64d3 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:54.906701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13bcbd-36d1-48d9-947b-ee3f381d64d3 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:54.912049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d933e-caa7-4725-aa6b-fe69b0394f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:54.912942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d933e-caa7-4725-aa6b-fe69b0394f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:54.913226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d933e-caa7-4725-aa6b-fe69b0394f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:54.913505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d933e-caa7-4725-aa6b-fe69b0394f8b admin admin] PATH_INFO: `/v3/projects/60b427c2601c400b97cb35a0e38a187f/users/8cb979e606ce4080a99c68e1e86339b8/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:54.928644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d933e-caa7-4725-aa6b-fe69b0394f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b427c2601c400b97cb35a0e38a187f, user_id=8cb979e606ce4080a99c68e1e86339b8, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:54.931602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d933e-caa7-4725-aa6b-fe69b0394f8b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.011250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1010/1997] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:54 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:55.015395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3603e7-a2ea-449d-a652-801921246fac admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.023090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 988/1998] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:54 2022] PUT /identity/v3/projects/60b427c2601c400b97cb35a0e38a187f/users/8cb979e606ce4080a99c68e1e86339b8/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:55.026891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3603e7-a2ea-449d-a652-801921246fac admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.035090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f0227-9472-459d-898b-a66438f66f69 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.035516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f0227-9472-459d-898b-a66438f66f69 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.035875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f0227-9472-459d-898b-a66438f66f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.063087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1e0f0227-9472-459d-898b-a66438f66f69 None None] MFA Rules not processed for user `8cb979e606ce4080a99c68e1e86339b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:55.064374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1e0f0227-9472-459d-898b-a66438f66f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:55.085598 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1637f-e2c3-42a3-8ec5-655af4b4f683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.086286 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1637f-e2c3-42a3-8ec5-655af4b4f683 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.086701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1637f-e2c3-42a3-8ec5-655af4b4f683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.086989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1637f-e2c3-42a3-8ec5-655af4b4f683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.087816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1637f-e2c3-42a3-8ec5-655af4b4f683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.089431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1637f-e2c3-42a3-8ec5-655af4b4f683 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.100207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1011/1999] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:55.104731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1637f-e2c3-42a3-8ec5-655af4b4f683 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.117787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1637f-e2c3-42a3-8ec5-655af4b4f683 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.123430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25ad7e-9ffb-4fa3-8a9e-367f61e90d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.124192 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25ad7e-9ffb-4fa3-8a9e-367f61e90d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.124550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25ad7e-9ffb-4fa3-8a9e-367f61e90d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.124904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25ad7e-9ffb-4fa3-8a9e-367f61e90d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.125719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25ad7e-9ffb-4fa3-8a9e-367f61e90d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.127425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25ad7e-9ffb-4fa3-8a9e-367f61e90d47 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.139085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1012/2000] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:55.143197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25ad7e-9ffb-4fa3-8a9e-367f61e90d47 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.153216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 989/2001] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:55.154197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25ad7e-9ffb-4fa3-8a9e-367f61e90d47 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.156598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f0227-9472-459d-898b-a66438f66f69 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.160646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c48113-6e79-4c04-9d1a-746cae6d3c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.161547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c48113-6e79-4c04-9d1a-746cae6d3c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.161832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c48113-6e79-4c04-9d1a-746cae6d3c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.162140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c48113-6e79-4c04-9d1a-746cae6d3c16 admin admin] PATH_INFO: `/v3/projects/5ca07372eccc4b4f9a3c90e3370be157/users/0f6cca49ee9a4004b0eacdb063aaa078/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.168154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f0227-9472-459d-898b-a66438f66f69 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.170400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:55.174756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec8aa-3231-4df8-bf79-9077c637fe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.175410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec8aa-3231-4df8-bf79-9077c637fe1d admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.180551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec8aa-3231-4df8-bf79-9077c637fe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.180551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec8aa-3231-4df8-bf79-9077c637fe1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.180551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ec8aa-3231-4df8-bf79-9077c637fe1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.180861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c48113-6e79-4c04-9d1a-746cae6d3c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca07372eccc4b4f9a3c90e3370be157, user_id=0f6cca49ee9a4004b0eacdb063aaa078, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.180955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ec8aa-3231-4df8-bf79-9077c637fe1d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.181037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c48113-6e79-4c04-9d1a-746cae6d3c16 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.257874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1013/2002] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:55 2022] PUT /identity/v3/projects/5ca07372eccc4b4f9a3c90e3370be157/users/0f6cca49ee9a4004b0eacdb063aaa078/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:55.268657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8833f9-96f1-41b7-99c0-3b85257b9927 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.269233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8833f9-96f1-41b7-99c0-3b85257b9927 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.269348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 990/2003] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:55 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:55.272668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8833f9-96f1-41b7-99c0-3b85257b9927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.273236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec8aa-3231-4df8-bf79-9077c637fe1d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.301426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ab8833f9-96f1-41b7-99c0-3b85257b9927 None None] MFA Rules not processed for user `0f6cca49ee9a4004b0eacdb063aaa078`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:55.303345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ab8833f9-96f1-41b7-99c0-3b85257b9927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:55.345441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec8aa-3231-4df8-bf79-9077c637fe1d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.350016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476a79b-25be-460b-996d-d18338ad7c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.350684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476a79b-25be-460b-996d-d18338ad7c00 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.350965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476a79b-25be-460b-996d-d18338ad7c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.351247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476a79b-25be-460b-996d-d18338ad7c00 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.351955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476a79b-25be-460b-996d-d18338ad7c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.353856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476a79b-25be-460b-996d-d18338ad7c00 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.354015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:55.369734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1014/2004] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:36:55.374341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8833f9-96f1-41b7-99c0-3b85257b9927 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.384925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8833f9-96f1-41b7-99c0-3b85257b9927 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.389508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17930b60-7998-419e-8bb1-337cc46618c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.390159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930b60-7998-419e-8bb1-337cc46618c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.390444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930b60-7998-419e-8bb1-337cc46618c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.390721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17930b60-7998-419e-8bb1-337cc46618c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.391437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17930b60-7998-419e-8bb1-337cc46618c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.393415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17930b60-7998-419e-8bb1-337cc46618c7 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.439210 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 991/2005] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:55 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:55.443407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476a79b-25be-460b-996d-d18338ad7c00 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.452987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476a79b-25be-460b-996d-d18338ad7c00 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.457585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06ac48-55e0-41ef-9b81-6598c86fe389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.458216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06ac48-55e0-41ef-9b81-6598c86fe389 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.458507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06ac48-55e0-41ef-9b81-6598c86fe389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.458785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06ac48-55e0-41ef-9b81-6598c86fe389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.460159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06ac48-55e0-41ef-9b81-6598c86fe389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.461715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06ac48-55e0-41ef-9b81-6598c86fe389 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.473050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 992/2006] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:55.477225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06ac48-55e0-41ef-9b81-6598c86fe389 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.486708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06ac48-55e0-41ef-9b81-6598c86fe389 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.491512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a75503-b72a-462a-a6d9-2b815fa7efb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.492381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a75503-b72a-462a-a6d9-2b815fa7efb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.492658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a75503-b72a-462a-a6d9-2b815fa7efb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.492933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a75503-b72a-462a-a6d9-2b815fa7efb4 admin admin] PATH_INFO: `/v3/projects/d0ed2d35994c4873a606f645455944d8/users/bef93cd35fcd412ab606913baf670c32/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.500353 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1015/2007] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:55 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:55.503171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17930b60-7998-419e-8bb1-337cc46618c7 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.509024 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a75503-b72a-462a-a6d9-2b815fa7efb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ed2d35994c4873a606f645455944d8, user_id=bef93cd35fcd412ab606913baf670c32, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.512344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a75503-b72a-462a-a6d9-2b815fa7efb4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.569905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17930b60-7998-419e-8bb1-337cc46618c7 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.575394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 993/2008] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:55 2022] PUT /identity/v3/projects/d0ed2d35994c4873a606f645455944d8/users/bef93cd35fcd412ab606913baf670c32/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:55.578344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a75503-b72a-462a-a6d9-2b815fa7efb4 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.627802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:55.631447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1211b-2d0c-4710-9d7f-5e9616d61da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.632154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1211b-2d0c-4710-9d7f-5e9616d61da5 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.632435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1211b-2d0c-4710-9d7f-5e9616d61da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.632712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1211b-2d0c-4710-9d7f-5e9616d61da5 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.633401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1211b-2d0c-4710-9d7f-5e9616d61da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.635422 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1211b-2d0c-4710-9d7f-5e9616d61da5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.644234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a75503-b72a-462a-a6d9-2b815fa7efb4 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.649973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec67a59-d870-4711-905e-2a9cc9aa0f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.651213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:55.651360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec67a59-d870-4711-905e-2a9cc9aa0f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.651360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec67a59-d870-4711-905e-2a9cc9aa0f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.651360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec67a59-d870-4711-905e-2a9cc9aa0f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.652040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec67a59-d870-4711-905e-2a9cc9aa0f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.653674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec67a59-d870-4711-905e-2a9cc9aa0f30 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.664497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 994/2009] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:36:55.668599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec67a59-d870-4711-905e-2a9cc9aa0f30 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.678727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec67a59-d870-4711-905e-2a9cc9aa0f30 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.683808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda4b41-71b1-49e8-b22e-e1aeeed5b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.684692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda4b41-71b1-49e8-b22e-e1aeeed5b431 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.684982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda4b41-71b1-49e8-b22e-e1aeeed5b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.685265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda4b41-71b1-49e8-b22e-e1aeeed5b431 admin admin] PATH_INFO: `/v3/projects/d0ed2d35994c4873a606f645455944d8/users/bef93cd35fcd412ab606913baf670c32/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.687339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda4b41-71b1-49e8-b22e-e1aeeed5b431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ed2d35994c4873a606f645455944d8, user_id=bef93cd35fcd412ab606913baf670c32, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.690104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda4b41-71b1-49e8-b22e-e1aeeed5b431 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.701935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 995/2010] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:55 2022] PUT /identity/v3/projects/d0ed2d35994c4873a606f645455944d8/users/bef93cd35fcd412ab606913baf670c32/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:55.705646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1016/2011] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:36:55 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:36:55.706354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda4b41-71b1-49e8-b22e-e1aeeed5b431 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.729899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215e6c5-50c3-49a0-b02b-5494a5eae79a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.731707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215e6c5-50c3-49a0-b02b-5494a5eae79a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.731707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215e6c5-50c3-49a0-b02b-5494a5eae79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.779704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8215e6c5-50c3-49a0-b02b-5494a5eae79a None None] MFA Rules not processed for user `bef93cd35fcd412ab606913baf670c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:55.781751 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8215e6c5-50c3-49a0-b02b-5494a5eae79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:55.788582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda4b41-71b1-49e8-b22e-e1aeeed5b431 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.793961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83fcec-9ee2-4a4b-afde-e71a9185f952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.794651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83fcec-9ee2-4a4b-afde-e71a9185f952 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.794932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83fcec-9ee2-4a4b-afde-e71a9185f952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.795199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83fcec-9ee2-4a4b-afde-e71a9185f952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.796018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83fcec-9ee2-4a4b-afde-e71a9185f952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.797976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83fcec-9ee2-4a4b-afde-e71a9185f952 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.814588 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 996/2012] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:55.819521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83fcec-9ee2-4a4b-afde-e71a9185f952 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.833101 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83fcec-9ee2-4a4b-afde-e71a9185f952 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.839468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd64839-ff6e-46bf-914a-161b57fa4c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.840563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd64839-ff6e-46bf-914a-161b57fa4c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.840937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd64839-ff6e-46bf-914a-161b57fa4c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.841305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd64839-ff6e-46bf-914a-161b57fa4c43 admin admin] PATH_INFO: `/v3/projects/4d735b05b9dd40e2b79e04ea22a56ad6/users/968477dcf559432297bd364181c8a69e/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.860251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd64839-ff6e-46bf-914a-161b57fa4c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d735b05b9dd40e2b79e04ea22a56ad6, user_id=968477dcf559432297bd364181c8a69e, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.863521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd64839-ff6e-46bf-914a-161b57fa4c43 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.886773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:55.901606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1017/2013] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:55.911053 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 997/2014] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:55 2022] PUT /identity/v3/projects/4d735b05b9dd40e2b79e04ea22a56ad6/users/968477dcf559432297bd364181c8a69e/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:55.920693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215e6c5-50c3-49a0-b02b-5494a5eae79a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.921043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd64839-ff6e-46bf-914a-161b57fa4c43 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:55.924228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6fd64839-ff6e-46bf-914a-161b57fa4c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:55.933885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215e6c5-50c3-49a0-b02b-5494a5eae79a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.942741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:55.995020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd64839-ff6e-46bf-914a-161b57fa4c43 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:55.996084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0e3d5-1bb9-4746-863b-c09c7626762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:55.996768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0e3d5-1bb9-4746-863b-c09c7626762f admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:55.997146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0e3d5-1bb9-4746-863b-c09c7626762f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:55.997434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0e3d5-1bb9-4746-863b-c09c7626762f admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:55.998166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0e3d5-1bb9-4746-863b-c09c7626762f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:55.999834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0e3d5-1bb9-4746-863b-c09c7626762f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:55.999938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:56.000594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:56.001089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:56.001419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:56.004990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:56.011106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1018/2015] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:36:56.015105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0e3d5-1bb9-4746-863b-c09c7626762f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:56.026075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0e3d5-1bb9-4746-863b-c09c7626762f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:56.031476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ed613-0af3-40dc-a47e-9cfe26a72222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:56.032559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ed613-0af3-40dc-a47e-9cfe26a72222 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:56.032933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ed613-0af3-40dc-a47e-9cfe26a72222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:56.033311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ed613-0af3-40dc-a47e-9cfe26a72222 admin admin] PATH_INFO: `/v3/projects/4d735b05b9dd40e2b79e04ea22a56ad6/users/968477dcf559432297bd364181c8a69e/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:56.036498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ed613-0af3-40dc-a47e-9cfe26a72222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d735b05b9dd40e2b79e04ea22a56ad6, user_id=968477dcf559432297bd364181c8a69e, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:56.040179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ed613-0af3-40dc-a47e-9cfe26a72222 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:56.054031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1019/2016] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:36:56 2022] PUT /identity/v3/projects/4d735b05b9dd40e2b79e04ea22a56ad6/users/968477dcf559432297bd364181c8a69e/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:36:56.066437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f55d4-31d2-488e-a685-45f9e6f18942 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:56.066676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f55d4-31d2-488e-a685-45f9e6f18942 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:56.067013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f55d4-31d2-488e-a685-45f9e6f18942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:56.072527 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:56.086728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:56.088231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:56.101213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 998/2017] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:56.104240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5f7f55d4-31d2-488e-a685-45f9e6f18942 None None] MFA Rules not processed for user `968477dcf559432297bd364181c8a69e`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:56.105536 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5f7f55d4-31d2-488e-a685-45f9e6f18942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:56.210681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1020/2018] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:56.213920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:56.261113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:56.263235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:56.286719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397798a8-9bfa-4d77-b0e9-83ce1d5254ab service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:56.352277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535dd92-672a-4da3-b04c-b35c793a1da0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:56.353115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535dd92-672a-4da3-b04c-b35c793a1da0 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:56.353385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535dd92-672a-4da3-b04c-b35c793a1da0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:56.353735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535dd92-672a-4da3-b04c-b35c793a1da0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:56.364522 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535dd92-672a-4da3-b04c-b35c793a1da0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:56.367038 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535dd92-672a-4da3-b04c-b35c793a1da0 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:56.377779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 999/2019] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:56.600078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:56.907323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f55d4-31d2-488e-a685-45f9e6f18942 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:56.923081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:56.923081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f55d4-31d2-488e-a685-45f9e6f18942 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:56.927271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83de50-334c-4ab6-97d0-4eceae9c0ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:56.928559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83de50-334c-4ab6-97d0-4eceae9c0ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:56.929052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83de50-334c-4ab6-97d0-4eceae9c0ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:56.929401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83de50-334c-4ab6-97d0-4eceae9c0ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:56.935162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9b83de50-334c-4ab6-97d0-4eceae9c0ed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:57.023739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83de50-334c-4ab6-97d0-4eceae9c0ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:57.025223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83de50-334c-4ab6-97d0-4eceae9c0ed1 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:57.036529 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1021/2020] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:57.493075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535dd92-672a-4da3-b04c-b35c793a1da0 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:57.503064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535dd92-672a-4da3-b04c-b35c793a1da0 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:57.508485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee02d3-95c1-42b7-b61b-4cde7c9134d9 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '968477dcf559432297bd364181c8a69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d735b05b9dd40e2b79e04ea22a56ad6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:57.509159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee02d3-95c1-42b7-b61b-4cde7c9134d9 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:57.509445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee02d3-95c1-42b7-b61b-4cde7c9134d9 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:57.511217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee02d3-95c1-42b7-b61b-4cde7c9134d9 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:57.511217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee02d3-95c1-42b7-b61b-4cde7c9134d9 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:57.511217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee02d3-95c1-42b7-b61b-4cde7c9134d9 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:57.512372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee02d3-95c1-42b7-b61b-4cde7c9134d9 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:57.528205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1000/2021] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:36:57.529449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83de50-334c-4ab6-97d0-4eceae9c0ed1 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:57.540330 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83de50-334c-4ab6-97d0-4eceae9c0ed1 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:57.544894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0004c-c90d-488c-84d1-3e48fe4cf3a3 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '968477dcf559432297bd364181c8a69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d735b05b9dd40e2b79e04ea22a56ad6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:57.545546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0004c-c90d-488c-84d1-3e48fe4cf3a3 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:57.545834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0004c-c90d-488c-84d1-3e48fe4cf3a3 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:57.546161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0004c-c90d-488c-84d1-3e48fe4cf3a3 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:57.548775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0004c-c90d-488c-84d1-3e48fe4cf3a3 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:57.548939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0004c-c90d-488c-84d1-3e48fe4cf3a3 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:57.647289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1022/2022] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:36:57 2022] POST /identity/v3/projects => generated 353 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:36:57.648125 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee02d3-95c1-42b7-b61b-4cde7c9134d9 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:57.658495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee02d3-95c1-42b7-b61b-4cde7c9134d9 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:57.691398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:57.714315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '968477dcf559432297bd364181c8a69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d735b05b9dd40e2b79e04ea22a56ad6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:57.714973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:57.715256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:57.715559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:57.716352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:36:57.716661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:57.718246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:57.736539 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1001/2023] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:36:57.960321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0004c-c90d-488c-84d1-3e48fe4cf3a3 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:57.977259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0004c-c90d-488c-84d1-3e48fe4cf3a3 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:57.982138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '968477dcf559432297bd364181c8a69e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d735b05b9dd40e2b79e04ea22a56ad6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:57.983366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:57.983904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:57.984401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] PATH_INFO: `/v3/projects/33a9ba22851f44b7b51809e5d7b03e3e` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:57.985817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Authorizing `identity:delete_project(project_id=33a9ba22851f44b7b51809e5d7b03e3e)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:57.987934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:58.140555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a9ba22851f44b7b51809e5d7b03e3e', 'request_id': 'req-3b24537a-3a7a-40af-8e31-b2de12040002'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:58.247345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a9ba22851f44b7b51809e5d7b03e3e', 'request_id': 'req-3b24537a-3a7a-40af-8e31-b2de12040002'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:58.256915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a9ba22851f44b7b51809e5d7b03e3e', 'request_id': 'req-3b24537a-3a7a-40af-8e31-b2de12040002'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:58.285634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] The token cache is being invalidate because project 33a9ba22851f44b7b51809e5d7b03e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:36:58.287893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:58.290488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:58.295961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1023/2024] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:36:57 2022] DELETE /identity/v3/projects/33a9ba22851f44b7b51809e5d7b03e3e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:58.734154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:58.736094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:58.795137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c692e-13e1-41f3-a341-373c67f36576 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:58.846896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c6261-1278-4ac2-85e9-5ce306abe603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:58.847645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c6261-1278-4ac2-85e9-5ce306abe603 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:58.847932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c6261-1278-4ac2-85e9-5ce306abe603 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:58.848214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c6261-1278-4ac2-85e9-5ce306abe603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:58.850235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-289c6261-1278-4ac2-85e9-5ce306abe603 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:58.915759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c6261-1278-4ac2-85e9-5ce306abe603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:58.917256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c6261-1278-4ac2-85e9-5ce306abe603 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:58.928377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1002/2025] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:36:59.401229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:59.403678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:59.473467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24537a-3a7a-40af-8e31-b2de12040002 tempest-VolumeQuotaClassesTest-944486666 tempest-VolumeQuotaClassesTest-944486666-project-admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:59.478936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:59.480014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:59.480322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:59.480597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] PATH_INFO: `/v3/users/0f6cca49ee9a4004b0eacdb063aaa078` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:59.481826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6cca49ee9a4004b0eacdb063aaa078)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:59.483890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:59.485597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:59.704071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6cca49ee9a4004b0eacdb063aaa078'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:59.781706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6cca49ee9a4004b0eacdb063aaa078'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:59.782323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6cca49ee9a4004b0eacdb063aaa078'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:59.790952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6cca49ee9a4004b0eacdb063aaa078'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:59.791735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6cca49ee9a4004b0eacdb063aaa078'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:36:59.810214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1024/2026] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:36:59 2022] DELETE /identity/v3/users/0f6cca49ee9a4004b0eacdb063aaa078 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:36:59.813936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4834f8b-193d-4241-af7d-ac6b004acfee None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:59.814386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4834f8b-193d-4241-af7d-ac6b004acfee None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:59.814560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4834f8b-193d-4241-af7d-ac6b004acfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:59.831381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d4834f8b-193d-4241-af7d-ac6b004acfee None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:36:59.832605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d4834f8b-193d-4241-af7d-ac6b004acfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:59.871346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:36:59.892008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1003/2027] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:36:59.902618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:36:59.904109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:36:59.961211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51e27e-ecf8-43ef-9cc7-0314cd354a17 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:36:59.964990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f920b1-a3a5-4fe8-b54f-97d79b39b35c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:36:59.965654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f920b1-a3a5-4fe8-b54f-97d79b39b35c service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:36:59.965946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f920b1-a3a5-4fe8-b54f-97d79b39b35c service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:36:59.966224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f920b1-a3a5-4fe8-b54f-97d79b39b35c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:36:59.976060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f920b1-a3a5-4fe8-b54f-97d79b39b35c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:36:59.977489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f920b1-a3a5-4fe8-b54f-97d79b39b35c service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:36:59.987264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1025/2028] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:00.326534 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4834f8b-193d-4241-af7d-ac6b004acfee None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:00.328105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d4834f8b-193d-4241-af7d-ac6b004acfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:00.342062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4834f8b-193d-4241-af7d-ac6b004acfee None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:00.348767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:00.348767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:00.348767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:00.348767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] PATH_INFO: `/v3/projects/5ca07372eccc4b4f9a3c90e3370be157` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:00.349800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca07372eccc4b4f9a3c90e3370be157)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:00.352122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:00.467343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca07372eccc4b4f9a3c90e3370be157', 'request_id': 'req-d081dc6b-0976-4cef-a552-42fc01dfd2f7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.472927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca07372eccc4b4f9a3c90e3370be157', 'request_id': 'req-d081dc6b-0976-4cef-a552-42fc01dfd2f7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.479174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca07372eccc4b4f9a3c90e3370be157', 'request_id': 'req-d081dc6b-0976-4cef-a552-42fc01dfd2f7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.546438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] The token cache is being invalidate because project 5ca07372eccc4b4f9a3c90e3370be157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:00.546729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.547358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.548955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1004/2029] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:00 2022] DELETE /identity/v3/projects/5ca07372eccc4b4f9a3c90e3370be157 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:00.553674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f920b1-a3a5-4fe8-b54f-97d79b39b35c service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:00.563888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f920b1-a3a5-4fe8-b54f-97d79b39b35c service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:00.622234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:00.623171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:00.623460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:00.623798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] PATH_INFO: `/v3/users/968477dcf559432297bd364181c8a69e` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:00.625025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=968477dcf559432297bd364181c8a69e)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:00.627006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:00.656387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:00.853525 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968477dcf559432297bd364181c8a69e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.935682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968477dcf559432297bd364181c8a69e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.936301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968477dcf559432297bd364181c8a69e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.943712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968477dcf559432297bd364181c8a69e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.944334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7eb0d5ca-ba71-4331-b9e2-8bbab21d6c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968477dcf559432297bd364181c8a69e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:00.963748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1026/2030] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:00 2022] DELETE /identity/v3/users/968477dcf559432297bd364181c8a69e => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:01.449576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:01.451018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:01.466497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081dc6b-0976-4cef-a552-42fc01dfd2f7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:01.527632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:01.528501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:01.528777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:01.529048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] PATH_INFO: `/v3/projects/4d735b05b9dd40e2b79e04ea22a56ad6` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:01.530285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d735b05b9dd40e2b79e04ea22a56ad6)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:01.532220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:01.638275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d735b05b9dd40e2b79e04ea22a56ad6', 'request_id': 'req-89c96949-d4ee-49b0-8388-bd12789ffa69'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:01.643644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d735b05b9dd40e2b79e04ea22a56ad6', 'request_id': 'req-89c96949-d4ee-49b0-8388-bd12789ffa69'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:01.650418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d735b05b9dd40e2b79e04ea22a56ad6', 'request_id': 'req-89c96949-d4ee-49b0-8388-bd12789ffa69'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:01.653942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:01.714287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] The token cache is being invalidate because project 4d735b05b9dd40e2b79e04ea22a56ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:01.714606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:01.715227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:01.717082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1005/2031] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:01 2022] DELETE /identity/v3/projects/4d735b05b9dd40e2b79e04ea22a56ad6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:01.726991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c46a31-f0ec-4fd3-91f1-b2157ac778bb None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:01.727436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c46a31-f0ec-4fd3-91f1-b2157ac778bb None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:01.727635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c46a31-f0ec-4fd3-91f1-b2157ac778bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:01.748702 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a6c46a31-f0ec-4fd3-91f1-b2157ac778bb None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:01.749689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a6c46a31-f0ec-4fd3-91f1-b2157ac778bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:01.814439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:01.816861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:01.829198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1027/2032] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:01.836951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93611c4e-02f1-4a4e-9fda-6fbffb0d174d None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:01.837271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93611c4e-02f1-4a4e-9fda-6fbffb0d174d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:01.837558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93611c4e-02f1-4a4e-9fda-6fbffb0d174d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:01.858449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-93611c4e-02f1-4a4e-9fda-6fbffb0d174d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:01.859695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-93611c4e-02f1-4a4e-9fda-6fbffb0d174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:01.868631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1028/2033] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:01.872382 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93611c4e-02f1-4a4e-9fda-6fbffb0d174d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:01.882852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93611c4e-02f1-4a4e-9fda-6fbffb0d174d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:01.891023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc91cd-a818-404e-ad72-40887ee78ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:01.891891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc91cd-a818-404e-ad72-40887ee78ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:01.892184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc91cd-a818-404e-ad72-40887ee78ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:01.892494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc91cd-a818-404e-ad72-40887ee78ceb admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:01.894552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc91cd-a818-404e-ad72-40887ee78ceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:01.894552 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc91cd-a818-404e-ad72-40887ee78ceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:01.896419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc91cd-a818-404e-ad72-40887ee78ceb admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:01.912819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1029/2034] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:37:01.919243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc91cd-a818-404e-ad72-40887ee78ceb admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:01.927217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c96949-d4ee-49b0-8388-bd12789ffa69 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:01.933972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc91cd-a818-404e-ad72-40887ee78ceb admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:01.935392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27a451-7232-42e6-bec0-697197866333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:01.936780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27a451-7232-42e6-bec0-697197866333 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:01.937236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27a451-7232-42e6-bec0-697197866333 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:01.937675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27a451-7232-42e6-bec0-697197866333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:01.939164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa627726-3088-440b-bfb7-abbc845f6e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:01.939921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa627726-3088-440b-bfb7-abbc845f6e12 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:01.940205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa627726-3088-440b-bfb7-abbc845f6e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:01.940643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa627726-3088-440b-bfb7-abbc845f6e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:01.941069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0b27a451-7232-42e6-bec0-697197866333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:01.949490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa627726-3088-440b-bfb7-abbc845f6e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:01.949490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa627726-3088-440b-bfb7-abbc845f6e12 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:02.016394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1030/2035] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:01 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:02.017640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa627726-3088-440b-bfb7-abbc845f6e12 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:02.020331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27a451-7232-42e6-bec0-697197866333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:02.023149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27a451-7232-42e6-bec0-697197866333 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:02.037576 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1006/2036] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:37:02.088670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa627726-3088-440b-bfb7-abbc845f6e12 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:02.093470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526e5b7-0fc7-468a-9427-be6337a19fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:02.094142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526e5b7-0fc7-468a-9427-be6337a19fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:02.094455 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526e5b7-0fc7-468a-9427-be6337a19fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:02.094735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526e5b7-0fc7-468a-9427-be6337a19fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:02.095431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526e5b7-0fc7-468a-9427-be6337a19fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:02.097401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526e5b7-0fc7-468a-9427-be6337a19fb4 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:02.104198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:02.194107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1031/2037] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:02 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:02.198339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27a451-7232-42e6-bec0-697197866333 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:02.209217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27a451-7232-42e6-bec0-697197866333 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:02.214070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f834f1-5dfd-4290-9bd3-aa4a8990f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:02.214766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f834f1-5dfd-4290-9bd3-aa4a8990f26e admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:02.216828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f834f1-5dfd-4290-9bd3-aa4a8990f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:02.216828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f834f1-5dfd-4290-9bd3-aa4a8990f26e admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:02.216828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f834f1-5dfd-4290-9bd3-aa4a8990f26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:02.217668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f834f1-5dfd-4290-9bd3-aa4a8990f26e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:02.229501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1007/2038] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:02.233856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526e5b7-0fc7-468a-9427-be6337a19fb4 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:02.245432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526e5b7-0fc7-468a-9427-be6337a19fb4 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:02.250363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad55b86-1eed-4357-b353-0aa7d19c4d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:02.251025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad55b86-1eed-4357-b353-0aa7d19c4d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:02.251311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad55b86-1eed-4357-b353-0aa7d19c4d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:02.251677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad55b86-1eed-4357-b353-0aa7d19c4d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:02.252611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad55b86-1eed-4357-b353-0aa7d19c4d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:02.254200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad55b86-1eed-4357-b353-0aa7d19c4d7d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:02.264873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1032/2039] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:02.269270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f834f1-5dfd-4290-9bd3-aa4a8990f26e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:02.283594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f834f1-5dfd-4290-9bd3-aa4a8990f26e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:02.297068 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd47a5-818f-4807-8f80-c33ce8057bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:02.298004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd47a5-818f-4807-8f80-c33ce8057bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:02.298432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd47a5-818f-4807-8f80-c33ce8057bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:02.298677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd47a5-818f-4807-8f80-c33ce8057bfa admin admin] PATH_INFO: `/v3/projects/d68fb6beb9f54676a62fc4e82d070232/users/33ea2971b9a34afeac0d3d2882d9e32e/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:02.324691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd47a5-818f-4807-8f80-c33ce8057bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d68fb6beb9f54676a62fc4e82d070232, user_id=33ea2971b9a34afeac0d3d2882d9e32e, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:02.329105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd47a5-818f-4807-8f80-c33ce8057bfa admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:02.419502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1008/2040] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:02 2022] PUT /identity/v3/projects/d68fb6beb9f54676a62fc4e82d070232/users/33ea2971b9a34afeac0d3d2882d9e32e/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:02.430427 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c6b1-341e-4943-92e1-c6b2e7a5bc35 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:02.430724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c6b1-341e-4943-92e1-c6b2e7a5bc35 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:02.431123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c6b1-341e-4943-92e1-c6b2e7a5bc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:02.475618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d4d3c6b1-341e-4943-92e1-c6b2e7a5bc35 None None] MFA Rules not processed for user `33ea2971b9a34afeac0d3d2882d9e32e`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:02.476864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d4d3c6b1-341e-4943-92e1-c6b2e7a5bc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:02.582611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1033/2041] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:02.584411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd47a5-818f-4807-8f80-c33ce8057bfa admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:02.655215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd47a5-818f-4807-8f80-c33ce8057bfa admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:02.661194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c5453-e13d-4565-a8f7-021e01ee183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:02.661885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c5453-e13d-4565-a8f7-021e01ee183d admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:02.662167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c5453-e13d-4565-a8f7-021e01ee183d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:02.662444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c5453-e13d-4565-a8f7-021e01ee183d admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:02.663142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c5453-e13d-4565-a8f7-021e01ee183d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:02.665378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c5453-e13d-4565-a8f7-021e01ee183d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:02.680142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:02.795309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1009/2042] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:02 2022] POST /identity/v3/projects => generated 367 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:02.799203 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3c6b1-341e-4943-92e1-c6b2e7a5bc35 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:02.810492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3c6b1-341e-4943-92e1-c6b2e7a5bc35 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:02.874949 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202800-e971-43f9-af45-cae8eb9afe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:02.875723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202800-e971-43f9-af45-cae8eb9afe70 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:02.876055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202800-e971-43f9-af45-cae8eb9afe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:02.876345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202800-e971-43f9-af45-cae8eb9afe70 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:02.877035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202800-e971-43f9-af45-cae8eb9afe70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:02.879356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202800-e971-43f9-af45-cae8eb9afe70 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:02.959726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1034/2043] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:02 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:02.964015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c5453-e13d-4565-a8f7-021e01ee183d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:02.975291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c5453-e13d-4565-a8f7-021e01ee183d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:02.980059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07c915-ca6c-436d-9998-f179fce19901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:02.986856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07c915-ca6c-436d-9998-f179fce19901 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:02.987150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07c915-ca6c-436d-9998-f179fce19901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:02.987437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07c915-ca6c-436d-9998-f179fce19901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:02.988279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07c915-ca6c-436d-9998-f179fce19901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:02.990887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07c915-ca6c-436d-9998-f179fce19901 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:03.001801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1010/2044] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:03.006145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202800-e971-43f9-af45-cae8eb9afe70 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:03.016237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202800-e971-43f9-af45-cae8eb9afe70 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:03.024962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc48e-e4bd-4879-ab06-c3dd04abfa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:03.025835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc48e-e4bd-4879-ab06-c3dd04abfa58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:03.026113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc48e-e4bd-4879-ab06-c3dd04abfa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:03.026384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc48e-e4bd-4879-ab06-c3dd04abfa58 admin admin] PATH_INFO: `/v3/projects/182b7021495d4daaa77c1974cb84e6bf/users/767d65d260354f08927f8f474e555055/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:03.041689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabc48e-e4bd-4879-ab06-c3dd04abfa58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182b7021495d4daaa77c1974cb84e6bf, user_id=767d65d260354f08927f8f474e555055, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:03.044383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabc48e-e4bd-4879-ab06-c3dd04abfa58 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:03.125572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1035/2045] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:03 2022] PUT /identity/v3/projects/182b7021495d4daaa77c1974cb84e6bf/users/767d65d260354f08927f8f474e555055/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:03.132382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07c915-ca6c-436d-9998-f179fce19901 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:03.148642 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07c915-ca6c-436d-9998-f179fce19901 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:03.211335 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535aaa1a-5a8c-4fe2-87f2-1543d8b864f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:03.212241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aaa1a-5a8c-4fe2-87f2-1543d8b864f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:03.212552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aaa1a-5a8c-4fe2-87f2-1543d8b864f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:03.212810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aaa1a-5a8c-4fe2-87f2-1543d8b864f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:03.213552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535aaa1a-5a8c-4fe2-87f2-1543d8b864f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:03.215224 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535aaa1a-5a8c-4fe2-87f2-1543d8b864f7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:03.230208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1011/2046] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:03.231414 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc48e-e4bd-4879-ab06-c3dd04abfa58 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:03.242707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc48e-e4bd-4879-ab06-c3dd04abfa58 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:03.247436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d46bc8-229a-4305-8fc3-80ecb617c7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:03.248382 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d46bc8-229a-4305-8fc3-80ecb617c7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:03.248665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d46bc8-229a-4305-8fc3-80ecb617c7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:03.248940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d46bc8-229a-4305-8fc3-80ecb617c7ae admin admin] PATH_INFO: `/v3/projects/182b7021495d4daaa77c1974cb84e6bf/users/767d65d260354f08927f8f474e555055/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:03.250979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d46bc8-229a-4305-8fc3-80ecb617c7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=182b7021495d4daaa77c1974cb84e6bf, user_id=767d65d260354f08927f8f474e555055, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:03.253753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d46bc8-229a-4305-8fc3-80ecb617c7ae admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:03.265642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1036/2047] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:03 2022] PUT /identity/v3/projects/182b7021495d4daaa77c1974cb84e6bf/users/767d65d260354f08927f8f474e555055/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:03.277903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade53f6-cdd9-473c-adcc-2f7e0f1fca06 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:03.278484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade53f6-cdd9-473c-adcc-2f7e0f1fca06 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:03.284010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade53f6-cdd9-473c-adcc-2f7e0f1fca06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:03.321169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4ade53f6-cdd9-473c-adcc-2f7e0f1fca06 None None] MFA Rules not processed for user `767d65d260354f08927f8f474e555055`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:03.322398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4ade53f6-cdd9-473c-adcc-2f7e0f1fca06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:03.440322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1012/2048] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:03.450441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d46bc8-229a-4305-8fc3-80ecb617c7ae admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:03.477867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:03.517017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d46bc8-229a-4305-8fc3-80ecb617c7ae admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:03.522028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633d8ff-7ddd-4813-bc43-f903c3fafd01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:03.522775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633d8ff-7ddd-4813-bc43-f903c3fafd01 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:03.523111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633d8ff-7ddd-4813-bc43-f903c3fafd01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:03.523493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633d8ff-7ddd-4813-bc43-f903c3fafd01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:03.533076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633d8ff-7ddd-4813-bc43-f903c3fafd01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:03.537541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633d8ff-7ddd-4813-bc43-f903c3fafd01 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:03.547066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1037/2049] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:04.515481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade53f6-cdd9-473c-adcc-2f7e0f1fca06 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:04.517105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4ade53f6-cdd9-473c-adcc-2f7e0f1fca06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:04.528967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade53f6-cdd9-473c-adcc-2f7e0f1fca06 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:04.532557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b1bc3-b61c-4bd8-a368-5284108278c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:04.533207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1bc3-b61c-4bd8-a368-5284108278c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:04.533490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1bc3-b61c-4bd8-a368-5284108278c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:04.533775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1bc3-b61c-4bd8-a368-5284108278c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:04.593857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b1bc3-b61c-4bd8-a368-5284108278c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:04.595330 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b1bc3-b61c-4bd8-a368-5284108278c5 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:04.605262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1013/2050] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:11.290716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633d8ff-7ddd-4813-bc43-f903c3fafd01 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:11.296582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9633d8ff-7ddd-4813-bc43-f903c3fafd01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:11.365892 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633d8ff-7ddd-4813-bc43-f903c3fafd01 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:11.371296 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:11.372287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:11.372637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:11.373003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] PATH_INFO: `/v3/users/8f789cfeb2d74ac999de56ed05f59d31` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:11.374297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f789cfeb2d74ac999de56ed05f59d31)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:11.376267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:11.584511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f789cfeb2d74ac999de56ed05f59d31'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:11.673450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f789cfeb2d74ac999de56ed05f59d31'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:11.674096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f789cfeb2d74ac999de56ed05f59d31'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:11.684212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f789cfeb2d74ac999de56ed05f59d31'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:11.685119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f789cfeb2d74ac999de56ed05f59d31'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:11.705297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1038/2051] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:11 2022] DELETE /identity/v3/users/8f789cfeb2d74ac999de56ed05f59d31 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:11.709876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3647a49-d21a-434f-852f-676377a00c83 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:11.710324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3647a49-d21a-434f-852f-676377a00c83 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:11.710568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3647a49-d21a-434f-852f-676377a00c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:11.718331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:11.730034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e3647a49-d21a-434f-852f-676377a00c83 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:11.731253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e3647a49-d21a-434f-852f-676377a00c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:11.794934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1014/2052] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:11.805256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:11.806682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:11.860530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:11.873522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5153d-f201-487c-858d-df5f97fb41bb admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:11.877264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a252e-df39-4716-ae72-096a10bca634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:11.878010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a252e-df39-4716-ae72-096a10bca634 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:11.878380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a252e-df39-4716-ae72-096a10bca634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:11.878728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a252e-df39-4716-ae72-096a10bca634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:11.889642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a252e-df39-4716-ae72-096a10bca634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:11.891105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a252e-df39-4716-ae72-096a10bca634 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:11.902118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1039/2053] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:12.606400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3647a49-d21a-434f-852f-676377a00c83 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:12.608575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e3647a49-d21a-434f-852f-676377a00c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:12.619971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3647a49-d21a-434f-852f-676377a00c83 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:12.626011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:12.626918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:12.627217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:12.627472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] PATH_INFO: `/v3/projects/409c153f006d4120a5fc925a25de409a` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:12.637803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=409c153f006d4120a5fc925a25de409a)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:12.637803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:12.765532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '409c153f006d4120a5fc925a25de409a', 'request_id': 'req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:12.774925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '409c153f006d4120a5fc925a25de409a', 'request_id': 'req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:12.779188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '409c153f006d4120a5fc925a25de409a', 'request_id': 'req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:12.794798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a252e-df39-4716-ae72-096a10bca634 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:12.796267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-620a252e-df39-4716-ae72-096a10bca634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:12.809259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a252e-df39-4716-ae72-096a10bca634 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:12.860888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] The token cache is being invalidate because project 409c153f006d4120a5fc925a25de409a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:12.861108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:12.861739 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:12.863340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1015/2054] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:12 2022] DELETE /identity/v3/projects/409c153f006d4120a5fc925a25de409a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:12.867085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:12.868646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:12.883874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:12.884766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:12.885031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:12.885334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] PATH_INFO: `/v3/users/33ea2971b9a34afeac0d3d2882d9e32e` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:12.886554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ea2971b9a34afeac0d3d2882d9e32e)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:12.888525 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:12.933950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9092d-c4f9-4b8e-958e-8389f3462e9e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:12.938730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:12.939748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:12.940053 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:12.940333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] PATH_INFO: `/v3/users/8cb979e606ce4080a99c68e1e86339b8` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:12.941546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb979e606ce4080a99c68e1e86339b8)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:12.944066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:13.040611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ea2971b9a34afeac0d3d2882d9e32e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.114091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:13.127826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb979e606ce4080a99c68e1e86339b8'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.128664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb979e606ce4080a99c68e1e86339b8'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.136203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb979e606ce4080a99c68e1e86339b8'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.136869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb979e606ce4080a99c68e1e86339b8'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.155834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb979e606ce4080a99c68e1e86339b8'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.176388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ea2971b9a34afeac0d3d2882d9e32e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.178875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ea2971b9a34afeac0d3d2882d9e32e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.185487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ea2971b9a34afeac0d3d2882d9e32e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.186300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-f3a2cae8-f3cb-4398-9af5-cb34c2644969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ea2971b9a34afeac0d3d2882d9e32e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.220856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1040/2055] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:12 2022] DELETE /identity/v3/users/33ea2971b9a34afeac0d3d2882d9e32e => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:13.222619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f98671-7971-4372-85ef-d6e97e3aa050 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:13.223156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f98671-7971-4372-85ef-d6e97e3aa050 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:13.223546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f98671-7971-4372-85ef-d6e97e3aa050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:13.242593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-59f98671-7971-4372-85ef-d6e97e3aa050 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:13.243868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-59f98671-7971-4372-85ef-d6e97e3aa050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:13.261831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1016/2056] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:12 2022] DELETE /identity/v3/users/8cb979e606ce4080a99c68e1e86339b8 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:13.313217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1041/2057] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:13.323453 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:13.325001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:13.382018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1544ce-acc4-43b6-94df-f91190fd29fc admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:13.385637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50839ca2-46ed-4f9f-9159-8b736ba1f22a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:13.386249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50839ca2-46ed-4f9f-9159-8b736ba1f22a service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:13.386529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50839ca2-46ed-4f9f-9159-8b736ba1f22a service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:13.386803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50839ca2-46ed-4f9f-9159-8b736ba1f22a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:13.445940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50839ca2-46ed-4f9f-9159-8b736ba1f22a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:13.447379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50839ca2-46ed-4f9f-9159-8b736ba1f22a service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:13.457213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1017/2058] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:13.732605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f98671-7971-4372-85ef-d6e97e3aa050 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:13.734120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-59f98671-7971-4372-85ef-d6e97e3aa050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:13.744787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:13.750217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f98671-7971-4372-85ef-d6e97e3aa050 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:13.756142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:13.756560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:13.759669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:13.759669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] PATH_INFO: `/v3/projects/60b427c2601c400b97cb35a0e38a187f` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:13.759669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b427c2601c400b97cb35a0e38a187f)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:13.760732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:13.884168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b427c2601c400b97cb35a0e38a187f', 'request_id': 'req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.954504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50839ca2-46ed-4f9f-9159-8b736ba1f22a service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:13.956271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-50839ca2-46ed-4f9f-9159-8b736ba1f22a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:13.969718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50839ca2-46ed-4f9f-9159-8b736ba1f22a service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:13.976551 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b427c2601c400b97cb35a0e38a187f', 'request_id': 'req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.984331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b427c2601c400b97cb35a0e38a187f', 'request_id': 'req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.996567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] The token cache is being invalidate because project 60b427c2601c400b97cb35a0e38a187f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:13.996841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.997500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:13.999285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1042/2059] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:13 2022] DELETE /identity/v3/projects/60b427c2601c400b97cb35a0e38a187f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:14.002800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:14.004933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:14.024841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:14.031334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:14.032307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:14.032606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:14.032874 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] PATH_INFO: `/v3/projects/d68fb6beb9f54676a62fc4e82d070232` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:14.034079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d68fb6beb9f54676a62fc4e82d070232)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:14.036006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:14.070577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2c80d-673f-40ba-9720-0ddaadbdfbf9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:14.075339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:14.076242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:14.076524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:14.076795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] PATH_INFO: `/v3/users/bef93cd35fcd412ab606913baf670c32` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:14.078002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef93cd35fcd412ab606913baf670c32)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:14.080008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:14.170903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd68fb6beb9f54676a62fc4e82d070232', 'request_id': 'req-57ad0f68-5fa2-472d-8775-86327d209254'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.176494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd68fb6beb9f54676a62fc4e82d070232', 'request_id': 'req-57ad0f68-5fa2-472d-8775-86327d209254'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.182599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd68fb6beb9f54676a62fc4e82d070232', 'request_id': 'req-57ad0f68-5fa2-472d-8775-86327d209254'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.254595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef93cd35fcd412ab606913baf670c32'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.271802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] The token cache is being invalidate because project d68fb6beb9f54676a62fc4e82d070232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:14.272250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.273199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.275416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1018/2060] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:13 2022] DELETE /identity/v3/projects/d68fb6beb9f54676a62fc4e82d070232 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:14.283201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:14.284599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:14.308814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef93cd35fcd412ab606913baf670c32'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.310277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef93cd35fcd412ab606913baf670c32'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.323657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef93cd35fcd412ab606913baf670c32'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.325583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef93cd35fcd412ab606913baf670c32'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.350601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1043/2061] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:14 2022] DELETE /identity/v3/users/bef93cd35fcd412ab606913baf670c32 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:14.361535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad0f68-5fa2-472d-8775-86327d209254 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:14.427511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:14.428484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:14.428848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:14.429133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] PATH_INFO: `/v3/users/767d65d260354f08927f8f474e555055` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:14.430370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=767d65d260354f08927f8f474e555055)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:14.432440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:14.610781 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:14.619744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '767d65d260354f08927f8f474e555055'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.620463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '767d65d260354f08927f8f474e555055'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.627285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '767d65d260354f08927f8f474e555055'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.627992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '767d65d260354f08927f8f474e555055'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.640309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e132989e-487a-4d31-8e36-a5a7e5000a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '767d65d260354f08927f8f474e555055'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:14.690165 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1019/2062] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:14 2022] DELETE /identity/v3/users/767d65d260354f08927f8f474e555055 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:14.829956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:14.831418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:14.844923 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea87a4f-d3c0-4501-aa9b-3735a6aa96c9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:14.849007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:14.913226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:14.913962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:14.914536 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:14.914812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] PATH_INFO: `/v3/projects/d0ed2d35994c4873a606f645455944d8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:14.916156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ed2d35994c4873a606f645455944d8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:14.918109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.024448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ed2d35994c4873a606f645455944d8', 'request_id': 'req-cfad0951-3c74-4944-9334-90f3ae4024ca'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.107883 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ed2d35994c4873a606f645455944d8', 'request_id': 'req-cfad0951-3c74-4944-9334-90f3ae4024ca'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.115035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ed2d35994c4873a606f645455944d8', 'request_id': 'req-cfad0951-3c74-4944-9334-90f3ae4024ca'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.126797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] The token cache is being invalidate because project d0ed2d35994c4873a606f645455944d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:15.127082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.127816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cfad0951-3c74-4944-9334-90f3ae4024ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.129757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1044/2063] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:14 2022] DELETE /identity/v3/projects/d0ed2d35994c4873a606f645455944d8 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:15.136671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be22e0-ed4e-4e9e-870a-7181d0b6a35e None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.141017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be22e0-ed4e-4e9e-870a-7181d0b6a35e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.141017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be22e0-ed4e-4e9e-870a-7181d0b6a35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.156104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e3be22e0-ed4e-4e9e-870a-7181d0b6a35e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:15.157326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e3be22e0-ed4e-4e9e-870a-7181d0b6a35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:15.217333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1020/2064] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:15.227197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae60d8f-5cf1-47b5-9994-408241df33c6 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.227197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae60d8f-5cf1-47b5-9994-408241df33c6 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.227197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae60d8f-5cf1-47b5-9994-408241df33c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.228977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be22e0-ed4e-4e9e-870a-7181d0b6a35e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.231006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e3be22e0-ed4e-4e9e-870a-7181d0b6a35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:15.243501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be22e0-ed4e-4e9e-870a-7181d0b6a35e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.248588 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.248976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-1ae60d8f-5cf1-47b5-9994-408241df33c6 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:15.249458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.249806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.250149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] PATH_INFO: `/v3/projects/182b7021495d4daaa77c1974cb84e6bf` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.250539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1ae60d8f-5cf1-47b5-9994-408241df33c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:15.251558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=182b7021495d4daaa77c1974cb84e6bf)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.253520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.261521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1045/2065] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:15.273855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae60d8f-5cf1-47b5-9994-408241df33c6 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.278010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae60d8f-5cf1-47b5-9994-408241df33c6 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.283137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5897703-cbb3-4683-9acf-565c74b16b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.283848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5897703-cbb3-4683-9acf-565c74b16b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.288207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5897703-cbb3-4683-9acf-565c74b16b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.288207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5897703-cbb3-4683-9acf-565c74b16b62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.288207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5897703-cbb3-4683-9acf-565c74b16b62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:15.288207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5897703-cbb3-4683-9acf-565c74b16b62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.288586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5897703-cbb3-4683-9acf-565c74b16b62 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.301532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1046/2066] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:37:15.306572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5897703-cbb3-4683-9acf-565c74b16b62 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.318437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5897703-cbb3-4683-9acf-565c74b16b62 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.327617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a1d00-66e7-45a7-b8c6-6291b0b7f394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.333921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a1d00-66e7-45a7-b8c6-6291b0b7f394 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.333921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a1d00-66e7-45a7-b8c6-6291b0b7f394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.333921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a1d00-66e7-45a7-b8c6-6291b0b7f394 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.333921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a1d00-66e7-45a7-b8c6-6291b0b7f394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.333921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a1d00-66e7-45a7-b8c6-6291b0b7f394 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.371163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182b7021495d4daaa77c1974cb84e6bf', 'request_id': 'req-30709226-f373-4648-941c-1ea7ae97be13'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.376679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182b7021495d4daaa77c1974cb84e6bf', 'request_id': 'req-30709226-f373-4648-941c-1ea7ae97be13'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.382855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182b7021495d4daaa77c1974cb84e6bf', 'request_id': 'req-30709226-f373-4648-941c-1ea7ae97be13'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.443379 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1047/2067] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:15 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:15.444407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a1d00-66e7-45a7-b8c6-6291b0b7f394 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.462922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] The token cache is being invalidate because project 182b7021495d4daaa77c1974cb84e6bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:15.463268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.463927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-30709226-f373-4648-941c-1ea7ae97be13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:15.465859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1021/2068] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:15 2022] DELETE /identity/v3/projects/182b7021495d4daaa77c1974cb84e6bf => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:15.470585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75401d12-bdce-4b70-98c8-4262f969b355 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.470880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75401d12-bdce-4b70-98c8-4262f969b355 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.471161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75401d12-bdce-4b70-98c8-4262f969b355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.491626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-75401d12-bdce-4b70-98c8-4262f969b355 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:15.492832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-75401d12-bdce-4b70-98c8-4262f969b355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:15.517656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a1d00-66e7-45a7-b8c6-6291b0b7f394 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.523821 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ed6bc-247d-483d-aa31-54828d010880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.524496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ed6bc-247d-483d-aa31-54828d010880 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.524781 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ed6bc-247d-483d-aa31-54828d010880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.525109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ed6bc-247d-483d-aa31-54828d010880 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.525803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ed6bc-247d-483d-aa31-54828d010880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.529095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ed6bc-247d-483d-aa31-54828d010880 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.538075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:15.567078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1022/2069] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:15.578382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00368c9-09fa-4f7e-ac6c-f615ad42954a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.579932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00368c9-09fa-4f7e-ac6c-f615ad42954a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.580217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00368c9-09fa-4f7e-ac6c-f615ad42954a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.601115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a00368c9-09fa-4f7e-ac6c-f615ad42954a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:15.602335 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a00368c9-09fa-4f7e-ac6c-f615ad42954a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:15.613151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1048/2070] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:15 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:15.614566 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ed6bc-247d-483d-aa31-54828d010880 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.616394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-324ed6bc-247d-483d-aa31-54828d010880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:15.619994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1023/2071] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:15.631582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00368c9-09fa-4f7e-ac6c-f615ad42954a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.631757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:15.634619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00368c9-09fa-4f7e-ac6c-f615ad42954a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.635390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ed6bc-247d-483d-aa31-54828d010880 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.640613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435b5fc-72bd-40c0-9a58-3498864e9eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.640758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9649b4-f53d-4bb3-bff0-0a70de2bc5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.640991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435b5fc-72bd-40c0-9a58-3498864e9eea admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.641157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9649b4-f53d-4bb3-bff0-0a70de2bc5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.641278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435b5fc-72bd-40c0-9a58-3498864e9eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.641492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9649b4-f53d-4bb3-bff0-0a70de2bc5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.641606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435b5fc-72bd-40c0-9a58-3498864e9eea admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.641869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9649b4-f53d-4bb3-bff0-0a70de2bc5c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.642267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435b5fc-72bd-40c0-9a58-3498864e9eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.642701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9649b4-f53d-4bb3-bff0-0a70de2bc5c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:15.643011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9649b4-f53d-4bb3-bff0-0a70de2bc5c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.643982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435b5fc-72bd-40c0-9a58-3498864e9eea admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.644680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9649b4-f53d-4bb3-bff0-0a70de2bc5c6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.659049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1049/2072] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:15.659221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1024/2073] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:37:15.660585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435b5fc-72bd-40c0-9a58-3498864e9eea admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.662630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9649b4-f53d-4bb3-bff0-0a70de2bc5c6 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.672213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435b5fc-72bd-40c0-9a58-3498864e9eea admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.673520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9649b4-f53d-4bb3-bff0-0a70de2bc5c6 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.677103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f104a-7ea7-4815-9758-0537b2184678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.677762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f104a-7ea7-4815-9758-0537b2184678 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.678143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f104a-7ea7-4815-9758-0537b2184678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.678401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f104a-7ea7-4815-9758-0537b2184678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.679129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef231-ef4b-4659-9605-1d34f2b6303b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.679268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f104a-7ea7-4815-9758-0537b2184678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.679870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef231-ef4b-4659-9605-1d34f2b6303b admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.682757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef231-ef4b-4659-9605-1d34f2b6303b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.682757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef231-ef4b-4659-9605-1d34f2b6303b admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.683131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f104a-7ea7-4815-9758-0537b2184678 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.683228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ef231-ef4b-4659-9605-1d34f2b6303b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.683228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ef231-ef4b-4659-9605-1d34f2b6303b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.691747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1050/2074] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:15.696043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f104a-7ea7-4815-9758-0537b2184678 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.706817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f104a-7ea7-4815-9758-0537b2184678 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.712465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20315326-b8a9-48b0-85e3-815d8e14c2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.713372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20315326-b8a9-48b0-85e3-815d8e14c2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.714714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20315326-b8a9-48b0-85e3-815d8e14c2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.714714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20315326-b8a9-48b0-85e3-815d8e14c2ec admin admin] PATH_INFO: `/v3/projects/9b392ccdc22844bd9cbfc48ffe81ba6b/users/50bd90c469d84beaaffc73cdbf413219/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.729162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20315326-b8a9-48b0-85e3-815d8e14c2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b392ccdc22844bd9cbfc48ffe81ba6b, user_id=50bd90c469d84beaaffc73cdbf413219, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.731901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20315326-b8a9-48b0-85e3-815d8e14c2ec admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.789962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1025/2075] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:15 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:15.794075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef231-ef4b-4659-9605-1d34f2b6303b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.810626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1051/2076] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:15 2022] PUT /identity/v3/projects/9b392ccdc22844bd9cbfc48ffe81ba6b/users/50bd90c469d84beaaffc73cdbf413219/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:15.826868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183a22-2c0d-4fc8-809e-5d3a1b0d2451 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.827159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183a22-2c0d-4fc8-809e-5d3a1b0d2451 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.827434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183a22-2c0d-4fc8-809e-5d3a1b0d2451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.855150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-25183a22-2c0d-4fc8-809e-5d3a1b0d2451 None None] MFA Rules not processed for user `50bd90c469d84beaaffc73cdbf413219`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:15.855696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef231-ef4b-4659-9605-1d34f2b6303b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:15.856438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-25183a22-2c0d-4fc8-809e-5d3a1b0d2451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:15.916945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce8f8-f0fe-4518-be89-9fe759f10bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:15.917824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce8f8-f0fe-4518-be89-9fe759f10bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:15.918105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce8f8-f0fe-4518-be89-9fe759f10bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:15.918382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ce8f8-f0fe-4518-be89-9fe759f10bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:15.919099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce8f8-f0fe-4518-be89-9fe759f10bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:15.921290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ce8f8-f0fe-4518-be89-9fe759f10bf7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:15.931555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1052/2077] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:15.943942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25183a22-2c0d-4fc8-809e-5d3a1b0d2451 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:15.945282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-25183a22-2c0d-4fc8-809e-5d3a1b0d2451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:15.999167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1026/2078] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:15 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:16.001131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25183a22-2c0d-4fc8-809e-5d3a1b0d2451 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:16.003594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce8f8-f0fe-4518-be89-9fe759f10bf7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:16.005619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a1a8-9bd1-47d4-bfb2-1517933d0058 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e33d1473077247cc97dcfd99b3da5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:16.006351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a1a8-9bd1-47d4-bfb2-1517933d0058 service glance] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:16.006655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a1a8-9bd1-47d4-bfb2-1517933d0058 service glance] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:16.006928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a1a8-9bd1-47d4-bfb2-1517933d0058 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:16.017798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ce8f8-f0fe-4518-be89-9fe759f10bf7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:16.017966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49a1a8-9bd1-47d4-bfb2-1517933d0058 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:16.017966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49a1a8-9bd1-47d4-bfb2-1517933d0058 service glance] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:16.018441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3098a40-b576-4e93-ba97-4aacefb8854b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:16.019089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3098a40-b576-4e93-ba97-4aacefb8854b admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:16.019372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3098a40-b576-4e93-ba97-4aacefb8854b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:16.019688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3098a40-b576-4e93-ba97-4aacefb8854b admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:16.020437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3098a40-b576-4e93-ba97-4aacefb8854b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:16.022625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3098a40-b576-4e93-ba97-4aacefb8854b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:16.026411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1053/2079] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:37:16.035602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1027/2080] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:16.036759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a1a8-9bd1-47d4-bfb2-1517933d0058 service glance] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:16.048799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a1a8-9bd1-47d4-bfb2-1517933d0058 service glance] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:16.053610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a0fe5-f913-4a3d-ab1a-0fc33fd368e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:16.054273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a0fe5-f913-4a3d-ab1a-0fc33fd368e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:16.054554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a0fe5-f913-4a3d-ab1a-0fc33fd368e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:16.054828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a0fe5-f913-4a3d-ab1a-0fc33fd368e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:16.055639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a0fe5-f913-4a3d-ab1a-0fc33fd368e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:16.057202 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a0fe5-f913-4a3d-ab1a-0fc33fd368e2 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:16.068308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1054/2081] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:16.072572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3098a40-b576-4e93-ba97-4aacefb8854b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:16.082918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3098a40-b576-4e93-ba97-4aacefb8854b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:16.088812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1dcc0-d5c0-4347-a68f-ab926af9904c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:16.089692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1dcc0-d5c0-4347-a68f-ab926af9904c admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:16.089969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1dcc0-d5c0-4347-a68f-ab926af9904c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:16.091623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1dcc0-d5c0-4347-a68f-ab926af9904c admin admin] PATH_INFO: `/v3/projects/cbb94df7abce44ddb826b8745bae1707/users/4c2805cc63ec44fdba477745e78119b0/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:16.107810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1dcc0-d5c0-4347-a68f-ab926af9904c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb94df7abce44ddb826b8745bae1707, user_id=4c2805cc63ec44fdba477745e78119b0, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:16.110571 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1dcc0-d5c0-4347-a68f-ab926af9904c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:16.205433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1028/2082] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:16 2022] PUT /identity/v3/projects/cbb94df7abce44ddb826b8745bae1707/users/4c2805cc63ec44fdba477745e78119b0/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:16.211167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbedc0-4388-4c18-9caf-9e79fce73be3 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:16.211601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbedc0-4388-4c18-9caf-9e79fce73be3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:16.211974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbedc0-4388-4c18-9caf-9e79fce73be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:16.230016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-86fbedc0-4388-4c18-9caf-9e79fce73be3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:16.231209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-86fbedc0-4388-4c18-9caf-9e79fce73be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:16.307804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1055/2083] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:16.314581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1dcc0-d5c0-4347-a68f-ab926af9904c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:16.320465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-00b1dcc0-d5c0-4347-a68f-ab926af9904c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:16.378785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1dcc0-d5c0-4347-a68f-ab926af9904c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:16.383023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e2375-fad2-4d7d-9cbc-49dd21a1208a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:16.383765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2375-fad2-4d7d-9cbc-49dd21a1208a service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:16.384045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2375-fad2-4d7d-9cbc-49dd21a1208a service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:16.384317 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2375-fad2-4d7d-9cbc-49dd21a1208a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:16.393555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e2375-fad2-4d7d-9cbc-49dd21a1208a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:16.395048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e2375-fad2-4d7d-9cbc-49dd21a1208a service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:16.404697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1029/2084] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:16.409916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:16.456053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:17.886211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e7436-701e-475e-b8ca-9aa0339decf0 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:17.886759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e7436-701e-475e-b8ca-9aa0339decf0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:17.886759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e7436-701e-475e-b8ca-9aa0339decf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:17.914739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5d5e7436-701e-475e-b8ca-9aa0339decf0 None None] MFA Rules not processed for user `4c2805cc63ec44fdba477745e78119b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:17.916084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5d5e7436-701e-475e-b8ca-9aa0339decf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:17.981726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1056/2085] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:17.986196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e2375-fad2-4d7d-9cbc-49dd21a1208a service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:17.996714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e2375-fad2-4d7d-9cbc-49dd21a1208a service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:18.001221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4e014-df73-43b6-b9eb-a2658ec25de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:18.001865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4e014-df73-43b6-b9eb-a2658ec25de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:18.002148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4e014-df73-43b6-b9eb-a2658ec25de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:18.002428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4e014-df73-43b6-b9eb-a2658ec25de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:18.003127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4e014-df73-43b6-b9eb-a2658ec25de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:18.005097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4e014-df73-43b6-b9eb-a2658ec25de6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:18.109406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1030/2086] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:17 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:18.113172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e7436-701e-475e-b8ca-9aa0339decf0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:18.125193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e7436-701e-475e-b8ca-9aa0339decf0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:18.185905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25069f81-67d7-42b2-ba7e-36be1dd13eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:18.186589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25069f81-67d7-42b2-ba7e-36be1dd13eed admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:18.186870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25069f81-67d7-42b2-ba7e-36be1dd13eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:18.187180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25069f81-67d7-42b2-ba7e-36be1dd13eed admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:18.187900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25069f81-67d7-42b2-ba7e-36be1dd13eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:18.189813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25069f81-67d7-42b2-ba7e-36be1dd13eed admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:18.266675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1057/2087] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:18 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:18.270675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4e014-df73-43b6-b9eb-a2658ec25de6 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:18.281572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4e014-df73-43b6-b9eb-a2658ec25de6 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:18.287788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca1703-9ea6-4bde-bdb9-f60aa823316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:18.288668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca1703-9ea6-4bde-bdb9-f60aa823316d admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:18.289050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca1703-9ea6-4bde-bdb9-f60aa823316d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:18.289695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca1703-9ea6-4bde-bdb9-f60aa823316d admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:18.290687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca1703-9ea6-4bde-bdb9-f60aa823316d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:18.292479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca1703-9ea6-4bde-bdb9-f60aa823316d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:18.304545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1031/2088] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:18.310177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25069f81-67d7-42b2-ba7e-36be1dd13eed admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:18.320286 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25069f81-67d7-42b2-ba7e-36be1dd13eed admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:18.324845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e21c7-8c28-48ad-8b3a-8341603454af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:18.325949 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e21c7-8c28-48ad-8b3a-8341603454af admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:18.326256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e21c7-8c28-48ad-8b3a-8341603454af admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:18.326503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e21c7-8c28-48ad-8b3a-8341603454af admin admin] PATH_INFO: `/v3/projects/c6581f8a6ea74ab7b7e89f09d97eebfe/users/8fe955dec19741bc89ed032422ad3ed5/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:18.354336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e21c7-8c28-48ad-8b3a-8341603454af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6581f8a6ea74ab7b7e89f09d97eebfe, user_id=8fe955dec19741bc89ed032422ad3ed5, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:18.357045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e21c7-8c28-48ad-8b3a-8341603454af admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:18.502228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1058/2089] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:18 2022] PUT /identity/v3/projects/c6581f8a6ea74ab7b7e89f09d97eebfe/users/8fe955dec19741bc89ed032422ad3ed5/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:18.510697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca1703-9ea6-4bde-bdb9-f60aa823316d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:18.521267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca1703-9ea6-4bde-bdb9-f60aa823316d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:18.577479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8a7f2-7768-4074-81fe-cdb2bb7f3846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:18.578163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8a7f2-7768-4074-81fe-cdb2bb7f3846 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:18.578427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8a7f2-7768-4074-81fe-cdb2bb7f3846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:18.578704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8a7f2-7768-4074-81fe-cdb2bb7f3846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:18.579426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8a7f2-7768-4074-81fe-cdb2bb7f3846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:18.581055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8a7f2-7768-4074-81fe-cdb2bb7f3846 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:18.592139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1032/2090] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:18.596514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e21c7-8c28-48ad-8b3a-8341603454af admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:18.607643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e21c7-8c28-48ad-8b3a-8341603454af admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:18.612511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8f280-4bef-4de6-b995-0ed7280ba93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:18.613326 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8f280-4bef-4de6-b995-0ed7280ba93c admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:18.613610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8f280-4bef-4de6-b995-0ed7280ba93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:18.613889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8f280-4bef-4de6-b995-0ed7280ba93c admin admin] PATH_INFO: `/v3/projects/c6581f8a6ea74ab7b7e89f09d97eebfe/users/8fe955dec19741bc89ed032422ad3ed5/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:18.615986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8f280-4bef-4de6-b995-0ed7280ba93c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6581f8a6ea74ab7b7e89f09d97eebfe, user_id=8fe955dec19741bc89ed032422ad3ed5, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:18.618651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8f280-4bef-4de6-b995-0ed7280ba93c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:18.628967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1059/2091] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:18 2022] PUT /identity/v3/projects/c6581f8a6ea74ab7b7e89f09d97eebfe/users/8fe955dec19741bc89ed032422ad3ed5/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:19.915327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb38a5f-1760-483b-9d65-2c4ea3b7d18f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:19.915881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb38a5f-1760-483b-9d65-2c4ea3b7d18f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:19.916080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb38a5f-1760-483b-9d65-2c4ea3b7d18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:19.952972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-5fb38a5f-1760-483b-9d65-2c4ea3b7d18f None None] MFA Rules not processed for user `8fe955dec19741bc89ed032422ad3ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:19.955001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5fb38a5f-1760-483b-9d65-2c4ea3b7d18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:20.008410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:20.028880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1033/2092] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:20.042285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8f280-4bef-4de6-b995-0ed7280ba93c admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:20.044180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ceb8f280-4bef-4de6-b995-0ed7280ba93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:20.050128 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:20.105474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8f280-4bef-4de6-b995-0ed7280ba93c admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:20.109685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161454-abc3-40a9-a1cb-df7805ca6a95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:20.110328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161454-abc3-40a9-a1cb-df7805ca6a95 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:20.110607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161454-abc3-40a9-a1cb-df7805ca6a95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:20.110878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb161454-abc3-40a9-a1cb-df7805ca6a95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:20.174292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb161454-abc3-40a9-a1cb-df7805ca6a95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:20.176115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb161454-abc3-40a9-a1cb-df7805ca6a95 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:20.185717 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1060/2093] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:27.121075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb38a5f-1760-483b-9d65-2c4ea3b7d18f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:27.122308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb38a5f-1760-483b-9d65-2c4ea3b7d18f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:27.126357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ff853-1ce7-47c0-9f62-d8d8edfbaa92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:27.127032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ff853-1ce7-47c0-9f62-d8d8edfbaa92 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:27.127320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ff853-1ce7-47c0-9f62-d8d8edfbaa92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:27.127631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ff853-1ce7-47c0-9f62-d8d8edfbaa92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:27.139048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ff853-1ce7-47c0-9f62-d8d8edfbaa92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:27.140630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ff853-1ce7-47c0-9f62-d8d8edfbaa92 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:27.155687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1034/2094] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:30.181558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=47 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:34.534501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161454-abc3-40a9-a1cb-df7805ca6a95 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:34.535997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-eb161454-abc3-40a9-a1cb-df7805ca6a95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:34.608280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb161454-abc3-40a9-a1cb-df7805ca6a95 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:34.613728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:34.614792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:34.614905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:34.615188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] PATH_INFO: `/v3/users/5a7eb45bf2c94759b3926b94f10c65b0` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:34.616599 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7eb45bf2c94759b3926b94f10c65b0)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:34.619103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:34.742357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b3384-80b9-48d8-8e6c-c8d038e0aeba None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:34.742889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b3384-80b9-48d8-8e6c-c8d038e0aeba None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:34.743089 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b3384-80b9-48d8-8e6c-c8d038e0aeba None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:34.744610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1035/2095] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:37:34.755414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53d852-2941-4f62-b4f9-53f06acf2055 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:34.756034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53d852-2941-4f62-b4f9-53f06acf2055 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:34.756034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53d852-2941-4f62-b4f9-53f06acf2055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:34.791172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ef53d852-2941-4f62-b4f9-53f06acf2055 None None] MFA Rules not processed for user `19d4d5fa82dc42e498fb22349921885e`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:34.792614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ef53d852-2941-4f62-b4f9-53f06acf2055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:34.824911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7eb45bf2c94759b3926b94f10c65b0'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:34.866540 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1036/2096] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:37:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:34.881851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53d852-2941-4f62-b4f9-53f06acf2055 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:34.883414 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ef53d852-2941-4f62-b4f9-53f06acf2055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:34.896994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53d852-2941-4f62-b4f9-53f06acf2055 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:34.904423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164c068-a92f-4794-ad87-cb67db9c13a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:34.904423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164c068-a92f-4794-ad87-cb67db9c13a3 service placement] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:34.904423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164c068-a92f-4794-ad87-cb67db9c13a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:34.904423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164c068-a92f-4794-ad87-cb67db9c13a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:34.918976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164c068-a92f-4794-ad87-cb67db9c13a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:34.920622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164c068-a92f-4794-ad87-cb67db9c13a3 service placement] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:34.924293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7eb45bf2c94759b3926b94f10c65b0'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:34.925143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7eb45bf2c94759b3926b94f10c65b0'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:34.933634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1037/2097] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:34.937707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7eb45bf2c94759b3926b94f10c65b0'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:34.938605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-791c4109-9ec0-4df4-b790-e8649114fdb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7eb45bf2c94759b3926b94f10c65b0'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:34.942959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164c068-a92f-4794-ad87-cb67db9c13a3 service placement] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:34.947223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6164c068-a92f-4794-ad87-cb67db9c13a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:34.965375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164c068-a92f-4794-ad87-cb67db9c13a3 service placement] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:34.976756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1061/2098] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:34 2022] DELETE /identity/v3/users/5a7eb45bf2c94759b3926b94f10c65b0 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:34.980063 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:34.981066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:34.981182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3cc3a-d95b-478d-8d1a-4a0939a125de None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:34.981354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:34.981450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3cc3a-d95b-478d-8d1a-4a0939a125de None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:34.981631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] PATH_INFO: `/v3/users/9b5d5a3f43c544368118125f2e194565` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:34.981737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3cc3a-d95b-478d-8d1a-4a0939a125de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:34.982880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5d5a3f43c544368118125f2e194565)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:34.984866 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:35.003030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-84f3cc3a-d95b-478d-8d1a-4a0939a125de None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:35.004376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-84f3cc3a-d95b-478d-8d1a-4a0939a125de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:35.080541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1062/2099] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:35.093699 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3cc3a-d95b-478d-8d1a-4a0939a125de None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:35.096000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-84f3cc3a-d95b-478d-8d1a-4a0939a125de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:35.119596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:35.164879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3cc3a-d95b-478d-8d1a-4a0939a125de None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:35.168674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff5843-162e-4a9e-ad43-07da893c713a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:35.169363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5843-162e-4a9e-ad43-07da893c713a service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:35.169667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5843-162e-4a9e-ad43-07da893c713a service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:35.169946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5843-162e-4a9e-ad43-07da893c713a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:35.179895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5843-162e-4a9e-ad43-07da893c713a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:35.181401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5843-162e-4a9e-ad43-07da893c713a service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:35.191858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1063/2100] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:37:35.260889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5d5a3f43c544368118125f2e194565'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:35.261839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5d5a3f43c544368118125f2e194565'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:35.280171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5d5a3f43c544368118125f2e194565'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:35.281116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5d5a3f43c544368118125f2e194565'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:35.302133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5d5a3f43c544368118125f2e194565'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:35.388421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1038/2101] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:34 2022] DELETE /identity/v3/users/9b5d5a3f43c544368118125f2e194565 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:35.680266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff5843-162e-4a9e-ad43-07da893c713a service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:35.682439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a2ff5843-162e-4a9e-ad43-07da893c713a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:35.696400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff5843-162e-4a9e-ad43-07da893c713a service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:35.760179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:35.761713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:35.765807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:35.766895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:35.767176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:35.767459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] PATH_INFO: `/v3/projects/84dd698260134176acd4f4892911ab05` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:35.770482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=84dd698260134176acd4f4892911ab05)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:35.772649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:35.803378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a22d4-e7ed-4d80-b9a0-22cdfaf18f8a admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:35.808795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:35.813914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:35.813914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:35.813914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] PATH_INFO: `/v3/projects/0b6930878a5c4f9b847362662c4d819b` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:35.813914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6930878a5c4f9b847362662c4d819b)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:35.813914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:35.910543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84dd698260134176acd4f4892911ab05', 'request_id': 'req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:35.925019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:35.945230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6930878a5c4f9b847362662c4d819b', 'request_id': 'req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:35.954388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6930878a5c4f9b847362662c4d819b', 'request_id': 'req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:35.961907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6930878a5c4f9b847362662c4d819b', 'request_id': 'req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:36.021306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84dd698260134176acd4f4892911ab05', 'request_id': 'req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:36.034657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84dd698260134176acd4f4892911ab05', 'request_id': 'req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:36.042150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] The token cache is being invalidate because project 84dd698260134176acd4f4892911ab05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:36.042440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:36.043742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9538a2d1-315e-4c0a-bfdb-fd03face5c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:36.047908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1064/2102] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:35 2022] DELETE /identity/v3/projects/84dd698260134176acd4f4892911ab05 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:36.052984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4da51-fc29-42ae-af5a-196d60c4afb5 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.053443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4da51-fc29-42ae-af5a-196d60c4afb5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.055051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4da51-fc29-42ae-af5a-196d60c4afb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.077570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7ac4da51-fc29-42ae-af5a-196d60c4afb5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:36.079074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7ac4da51-fc29-42ae-af5a-196d60c4afb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:36.132958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:36.135969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] The token cache is being invalidate because project 0b6930878a5c4f9b847362662c4d819b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:36.136654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:36.137450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-00bb593d-355b-4bfd-96b9-1d3a26e47f3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:36.139189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1039/2103] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:35 2022] DELETE /identity/v3/projects/0b6930878a5c4f9b847362662c4d819b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:36.144182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70274a-0c49-44a6-a45d-3a760da9c9d1 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.144737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70274a-0c49-44a6-a45d-3a760da9c9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.145226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70274a-0c49-44a6-a45d-3a760da9c9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.154797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1065/2104] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:36.161356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e8877-1a17-42db-86da-d34f7f6dcfb8 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.161656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e8877-1a17-42db-86da-d34f7f6dcfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.161935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e8877-1a17-42db-86da-d34f7f6dcfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.167618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ac70274a-0c49-44a6-a45d-3a760da9c9d1 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:36.168828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ac70274a-0c49-44a6-a45d-3a760da9c9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:36.190682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1040/2105] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:36.190801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-361e8877-1a17-42db-86da-d34f7f6dcfb8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:36.190801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-361e8877-1a17-42db-86da-d34f7f6dcfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:36.193011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e85cf-c626-405e-acdf-8c2bea684ba8 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.193308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e85cf-c626-405e-acdf-8c2bea684ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.193589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e85cf-c626-405e-acdf-8c2bea684ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.195513 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1066/2106] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:36.200329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e8877-1a17-42db-86da-d34f7f6dcfb8 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.213685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-2d5e85cf-c626-405e-acdf-8c2bea684ba8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:36.214901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2d5e85cf-c626-405e-acdf-8c2bea684ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:36.215777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e8877-1a17-42db-86da-d34f7f6dcfb8 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.221748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac92238-3e53-429e-aa29-1fc87af087c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.222577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac92238-3e53-429e-aa29-1fc87af087c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.222742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac92238-3e53-429e-aa29-1fc87af087c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.223031 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac92238-3e53-429e-aa29-1fc87af087c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.223517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1041/2107] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:36.223856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac92238-3e53-429e-aa29-1fc87af087c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:36.224169 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac92238-3e53-429e-aa29-1fc87af087c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.226409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac92238-3e53-429e-aa29-1fc87af087c0 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.228437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e85cf-c626-405e-acdf-8c2bea684ba8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.237828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e85cf-c626-405e-acdf-8c2bea684ba8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.243813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd200a4-ef22-44ad-ac76-0c11858c46b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.244682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd200a4-ef22-44ad-ac76-0c11858c46b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.245030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1067/2108] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:37:36.245327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd200a4-ef22-44ad-ac76-0c11858c46b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.245606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd200a4-ef22-44ad-ac76-0c11858c46b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.246342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd200a4-ef22-44ad-ac76-0c11858c46b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:36.246650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd200a4-ef22-44ad-ac76-0c11858c46b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.248376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd200a4-ef22-44ad-ac76-0c11858c46b9 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.256510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac92238-3e53-429e-aa29-1fc87af087c0 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.269332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac92238-3e53-429e-aa29-1fc87af087c0 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.274242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1042/2109] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:37:36.275248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca82f69-f1dc-49df-b217-b179015e2140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.275940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca82f69-f1dc-49df-b217-b179015e2140 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.276221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca82f69-f1dc-49df-b217-b179015e2140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.276495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca82f69-f1dc-49df-b217-b179015e2140 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.277496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca82f69-f1dc-49df-b217-b179015e2140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.279342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd200a4-ef22-44ad-ac76-0c11858c46b9 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.281476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca82f69-f1dc-49df-b217-b179015e2140 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.300275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd200a4-ef22-44ad-ac76-0c11858c46b9 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.312606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa242bb-f436-425d-abf6-4c334eaef56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.313913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa242bb-f436-425d-abf6-4c334eaef56b admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.314393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa242bb-f436-425d-abf6-4c334eaef56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.314668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa242bb-f436-425d-abf6-4c334eaef56b admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.315364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa242bb-f436-425d-abf6-4c334eaef56b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.317926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa242bb-f436-425d-abf6-4c334eaef56b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.383546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1068/2110] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:36.387492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca82f69-f1dc-49df-b217-b179015e2140 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.419393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1043/2111] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:36.422729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa242bb-f436-425d-abf6-4c334eaef56b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.450440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca82f69-f1dc-49df-b217-b179015e2140 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.484240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa242bb-f436-425d-abf6-4c334eaef56b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.492705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20469791-ef7e-49cf-bfe1-6b054ecbcb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.493501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20469791-ef7e-49cf-bfe1-6b054ecbcb02 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.493976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20469791-ef7e-49cf-bfe1-6b054ecbcb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.494762 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20469791-ef7e-49cf-bfe1-6b054ecbcb02 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.495739 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20469791-ef7e-49cf-bfe1-6b054ecbcb02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.498030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20469791-ef7e-49cf-bfe1-6b054ecbcb02 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.515621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00263c6c-4b01-40de-aea7-df441f111751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.516637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00263c6c-4b01-40de-aea7-df441f111751 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.516922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00263c6c-4b01-40de-aea7-df441f111751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.517204 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00263c6c-4b01-40de-aea7-df441f111751 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.517937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00263c6c-4b01-40de-aea7-df441f111751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.519910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00263c6c-4b01-40de-aea7-df441f111751 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.615672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1044/2112] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:36.620329 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20469791-ef7e-49cf-bfe1-6b054ecbcb02 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.628722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1069/2113] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:36.630261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20469791-ef7e-49cf-bfe1-6b054ecbcb02 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.634755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00263c6c-4b01-40de-aea7-df441f111751 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.636357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757ad9-b39e-4651-b83b-c5f3530e935e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.637016 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757ad9-b39e-4651-b83b-c5f3530e935e admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.637300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757ad9-b39e-4651-b83b-c5f3530e935e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.637578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757ad9-b39e-4651-b83b-c5f3530e935e admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.638310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a757ad9-b39e-4651-b83b-c5f3530e935e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.639982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a757ad9-b39e-4651-b83b-c5f3530e935e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.644887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00263c6c-4b01-40de-aea7-df441f111751 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.650249 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ebcc9-c9b5-4f7f-b763-31f906092ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.650430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1045/2114] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:36.651243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ebcc9-c9b5-4f7f-b763-31f906092ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.651555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ebcc9-c9b5-4f7f-b763-31f906092ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.651844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ebcc9-c9b5-4f7f-b763-31f906092ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.652639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ebcc9-c9b5-4f7f-b763-31f906092ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.654379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757ad9-b39e-4651-b83b-c5f3530e935e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.654495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ebcc9-c9b5-4f7f-b763-31f906092ca5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.665447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757ad9-b39e-4651-b83b-c5f3530e935e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.666688 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1070/2115] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:36.670267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f09cf-1ba0-49f0-ba99-c70a3f7ef95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.670955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f09cf-1ba0-49f0-ba99-c70a3f7ef95a admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.671260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f09cf-1ba0-49f0-ba99-c70a3f7ef95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.671561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f09cf-1ba0-49f0-ba99-c70a3f7ef95a admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.672289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ebcc9-c9b5-4f7f-b763-31f906092ca5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.672398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f09cf-1ba0-49f0-ba99-c70a3f7ef95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.673973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f09cf-1ba0-49f0-ba99-c70a3f7ef95a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.682928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ebcc9-c9b5-4f7f-b763-31f906092ca5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.689506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1046/2116] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:36.689629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351fb68-118c-445f-b151-967eb0cb7659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.689629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351fb68-118c-445f-b151-967eb0cb7659 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.689629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351fb68-118c-445f-b151-967eb0cb7659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.689629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351fb68-118c-445f-b151-967eb0cb7659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.690212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351fb68-118c-445f-b151-967eb0cb7659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.691094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f09cf-1ba0-49f0-ba99-c70a3f7ef95a admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.691856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351fb68-118c-445f-b151-967eb0cb7659 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.701126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f09cf-1ba0-49f0-ba99-c70a3f7ef95a admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.704015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1071/2117] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:36.708170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351fb68-118c-445f-b151-967eb0cb7659 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.710344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228b032-9e9f-4799-8db7-db68a8e2058c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.711258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228b032-9e9f-4799-8db7-db68a8e2058c admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.711581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228b032-9e9f-4799-8db7-db68a8e2058c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.711845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228b032-9e9f-4799-8db7-db68a8e2058c admin admin] PATH_INFO: `/v3/projects/e301883825434ae0b68261b3dae4a323/users/c08d0dba3e4442eda23f81261e2d40c2/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.718314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351fb68-118c-445f-b151-967eb0cb7659 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.723013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6e336-4d9f-4eec-8f49-c19b359fddcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:36.723920 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6e336-4d9f-4eec-8f49-c19b359fddcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.724199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6e336-4d9f-4eec-8f49-c19b359fddcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.724472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6e336-4d9f-4eec-8f49-c19b359fddcc admin admin] PATH_INFO: `/v3/projects/4b698ead017b4ef8b397ee127b487309/users/cc8a087e97424d2890952be33695378f/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.728673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228b032-9e9f-4799-8db7-db68a8e2058c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e301883825434ae0b68261b3dae4a323, user_id=c08d0dba3e4442eda23f81261e2d40c2, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.731432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228b032-9e9f-4799-8db7-db68a8e2058c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.739397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6e336-4d9f-4eec-8f49-c19b359fddcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b698ead017b4ef8b397ee127b487309, user_id=cc8a087e97424d2890952be33695378f, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:36.742873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6e336-4d9f-4eec-8f49-c19b359fddcc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:36.805330 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1047/2118] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:36 2022] PUT /identity/v3/projects/e301883825434ae0b68261b3dae4a323/users/c08d0dba3e4442eda23f81261e2d40c2/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:36.815662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119b238-f5e5-4494-ab63-4575034974ec None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.816144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119b238-f5e5-4494-ab63-4575034974ec None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.816296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119b238-f5e5-4494-ab63-4575034974ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.829659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1072/2119] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:36 2022] PUT /identity/v3/projects/4b698ead017b4ef8b397ee127b487309/users/cc8a087e97424d2890952be33695378f/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:36.834935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece84cf-74a1-4be3-8f88-f8cef4623859 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:36.835353 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece84cf-74a1-4be3-8f88-f8cef4623859 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:36.835734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece84cf-74a1-4be3-8f88-f8cef4623859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:36.840388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:36.842998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:36.849306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8119b238-f5e5-4494-ab63-4575034974ec None None] MFA Rules not processed for user `c08d0dba3e4442eda23f81261e2d40c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:36.850585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8119b238-f5e5-4494-ab63-4575034974ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:36.856593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-bece84cf-74a1-4be3-8f88-f8cef4623859 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:36.857851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-bece84cf-74a1-4be3-8f88-f8cef4623859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:36.923952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1048/2120] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:36.925414 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1073/2121] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:36.936773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece84cf-74a1-4be3-8f88-f8cef4623859 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.936910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119b238-f5e5-4494-ab63-4575034974ec None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:36.938164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-bece84cf-74a1-4be3-8f88-f8cef4623859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:36.938534 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8119b238-f5e5-4494-ab63-4575034974ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:36.996421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece84cf-74a1-4be3-8f88-f8cef4623859 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:36.996785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119b238-f5e5-4494-ab63-4575034974ec None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:37.000198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd0fea-3d38-4dd3-9027-bec967adf5dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:37.000839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd0fea-3d38-4dd3-9027-bec967adf5dc service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:37.000942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be9bbf-a4a9-46fc-b81d-72cb25065bae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:37.001133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd0fea-3d38-4dd3-9027-bec967adf5dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:37.001398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd0fea-3d38-4dd3-9027-bec967adf5dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:37.001561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9bbf-a4a9-46fc-b81d-72cb25065bae service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:37.001847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9bbf-a4a9-46fc-b81d-72cb25065bae service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:37.002220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9bbf-a4a9-46fc-b81d-72cb25065bae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:37.011324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd0fea-3d38-4dd3-9027-bec967adf5dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:37.013366 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be9bbf-a4a9-46fc-b81d-72cb25065bae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:37.013473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd0fea-3d38-4dd3-9027-bec967adf5dc service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:37.013652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be9bbf-a4a9-46fc-b81d-72cb25065bae service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:37.024161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1074/2122] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:37.024534 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1049/2123] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:38.669504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd0fea-3d38-4dd3-9027-bec967adf5dc service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:38.671021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-dadd0fea-3d38-4dd3-9027-bec967adf5dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:38.688070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd0fea-3d38-4dd3-9027-bec967adf5dc service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:38.695083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:38.696832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:38.697098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:38.697690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] PATH_INFO: `/v3/users/2338f10adaad475d9bd735547c593494` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:38.699958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2338f10adaad475d9bd735547c593494)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:38.702357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:38.923577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2338f10adaad475d9bd735547c593494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:38.998707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2338f10adaad475d9bd735547c593494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:38.999497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2338f10adaad475d9bd735547c593494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:39.008855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2338f10adaad475d9bd735547c593494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:39.009781 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2338f10adaad475d9bd735547c593494'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:39.038103 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1075/2124] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:38 2022] DELETE /identity/v3/users/2338f10adaad475d9bd735547c593494 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:39.043447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823eeb6-8761-4aa0-b86a-531b6f7108f5 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:39.043965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823eeb6-8761-4aa0-b86a-531b6f7108f5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:39.044082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823eeb6-8761-4aa0-b86a-531b6f7108f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:39.064290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a823eeb6-8761-4aa0-b86a-531b6f7108f5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:39.069289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a823eeb6-8761-4aa0-b86a-531b6f7108f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:39.143945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1050/2125] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:39.154138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:39.155808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:39.185052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:39.226741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf643de7-2e53-436f-9da6-2de921ea2234 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:39.230678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ee689-4888-4514-8645-b9411bcc9acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:39.231460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ee689-4888-4514-8645-b9411bcc9acc service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:39.231816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ee689-4888-4514-8645-b9411bcc9acc service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:39.232147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ee689-4888-4514-8645-b9411bcc9acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:39.242611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ee689-4888-4514-8645-b9411bcc9acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:39.244206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ee689-4888-4514-8645-b9411bcc9acc service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:39.255472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1076/2126] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:37:39.830562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823eeb6-8761-4aa0-b86a-531b6f7108f5 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:39.832591 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a823eeb6-8761-4aa0-b86a-531b6f7108f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:39.860257 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823eeb6-8761-4aa0-b86a-531b6f7108f5 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:39.878905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:39.880656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:39.880656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:39.880656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] PATH_INFO: `/v3/projects/467854d7e6a4465baea90f2e2b41ca5b` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:39.882352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=467854d7e6a4465baea90f2e2b41ca5b)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:39.884326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:40.025045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:40.034086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467854d7e6a4465baea90f2e2b41ca5b', 'request_id': 'req-ed7f8a5f-1303-43d8-9643-426b95e4a468'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.040905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467854d7e6a4465baea90f2e2b41ca5b', 'request_id': 'req-ed7f8a5f-1303-43d8-9643-426b95e4a468'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.047175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467854d7e6a4465baea90f2e2b41ca5b', 'request_id': 'req-ed7f8a5f-1303-43d8-9643-426b95e4a468'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.193048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] The token cache is being invalidate because project 467854d7e6a4465baea90f2e2b41ca5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:40.193400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.195755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.197578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1051/2127] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:39 2022] DELETE /identity/v3/projects/467854d7e6a4465baea90f2e2b41ca5b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:40.203034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ee689-4888-4514-8645-b9411bcc9acc service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:40.213493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ee689-4888-4514-8645-b9411bcc9acc service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:40.283004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:40.284201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:40.284675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:40.285131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] PATH_INFO: `/v3/users/f91ef3807664477fafe7f40c7412e62a` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:40.287112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f91ef3807664477fafe7f40c7412e62a)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:40.289712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:40.522944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f91ef3807664477fafe7f40c7412e62a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.604794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f91ef3807664477fafe7f40c7412e62a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.605490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f91ef3807664477fafe7f40c7412e62a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.619613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f91ef3807664477fafe7f40c7412e62a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.620882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9a1e4e3b-2779-4e64-baf5-a29dc910bcf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f91ef3807664477fafe7f40c7412e62a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:40.658849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1077/2128] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:40 2022] DELETE /identity/v3/users/f91ef3807664477fafe7f40c7412e62a => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:41.286544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:41.292278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:41.318312 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f8a5f-1303-43d8-9643-426b95e4a468 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:41.392612 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:41.393856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:41.394134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:41.394410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] PATH_INFO: `/v3/projects/4bfe924bf9c24328b9a3c346769fb942` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:41.395784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bfe924bf9c24328b9a3c346769fb942)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:41.398156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:41.562626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bfe924bf9c24328b9a3c346769fb942', 'request_id': 'req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:41.585993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bfe924bf9c24328b9a3c346769fb942', 'request_id': 'req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:41.610891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bfe924bf9c24328b9a3c346769fb942', 'request_id': 'req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:41.684574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] The token cache is being invalidate because project 4bfe924bf9c24328b9a3c346769fb942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:41.691799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:41.691799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a7b6aec6-e4a7-487a-a532-a2a6f8f509e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:41.691799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1052/2129] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:41 2022] DELETE /identity/v3/projects/4bfe924bf9c24328b9a3c346769fb942 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:41.693120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce9de7-cee7-46d2-9416-5d19747fcf1c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:41.693600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce9de7-cee7-46d2-9416-5d19747fcf1c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:41.693864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce9de7-cee7-46d2-9416-5d19747fcf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:41.713244 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7fce9de7-cee7-46d2-9416-5d19747fcf1c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:41.714643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7fce9de7-cee7-46d2-9416-5d19747fcf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:41.755309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:41.789993 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1078/2130] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:41.798764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9b0ec-50c2-4408-9497-f277e33f1f3b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:41.799056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9b0ec-50c2-4408-9497-f277e33f1f3b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:41.799338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9b0ec-50c2-4408-9497-f277e33f1f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:41.819367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f9d9b0ec-50c2-4408-9497-f277e33f1f3b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:41.820710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f9d9b0ec-50c2-4408-9497-f277e33f1f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:41.830248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1053/2131] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:41.834548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce9de7-cee7-46d2-9416-5d19747fcf1c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:41.846267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce9de7-cee7-46d2-9416-5d19747fcf1c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:41.852056 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d124-308e-4b66-a7ec-7b51031eff7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:41.852910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d124-308e-4b66-a7ec-7b51031eff7b admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:41.853310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d124-308e-4b66-a7ec-7b51031eff7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:41.853808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d124-308e-4b66-a7ec-7b51031eff7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:41.854756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d124-308e-4b66-a7ec-7b51031eff7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:41.855264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d124-308e-4b66-a7ec-7b51031eff7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:41.857087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d124-308e-4b66-a7ec-7b51031eff7b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:41.875758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1079/2132] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:37:41.881585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9b0ec-50c2-4408-9497-f277e33f1f3b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:41.893311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9b0ec-50c2-4408-9497-f277e33f1f3b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:41.900307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35a41b-6b1b-47cc-b906-e1eecc45128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:41.900978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35a41b-6b1b-47cc-b906-e1eecc45128c admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:41.901423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35a41b-6b1b-47cc-b906-e1eecc45128c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:41.901720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35a41b-6b1b-47cc-b906-e1eecc45128c admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:41.903181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35a41b-6b1b-47cc-b906-e1eecc45128c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:41.905780 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35a41b-6b1b-47cc-b906-e1eecc45128c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:41.993548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1054/2133] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:41 2022] POST /identity/v3/projects => generated 393 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:41.997569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d124-308e-4b66-a7ec-7b51031eff7b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:42.013262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d124-308e-4b66-a7ec-7b51031eff7b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:42.098970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b04f7d-9bc9-4db5-8f10-de6cb020f317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:42.099690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b04f7d-9bc9-4db5-8f10-de6cb020f317 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:42.099970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b04f7d-9bc9-4db5-8f10-de6cb020f317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:42.100407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b04f7d-9bc9-4db5-8f10-de6cb020f317 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:42.101137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b04f7d-9bc9-4db5-8f10-de6cb020f317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:42.103118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b04f7d-9bc9-4db5-8f10-de6cb020f317 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:42.185101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1080/2134] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:41 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:42.187719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35a41b-6b1b-47cc-b906-e1eecc45128c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:42.230359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35a41b-6b1b-47cc-b906-e1eecc45128c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:42.234632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61b915-9eb9-4c2a-a429-3936a64f59df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:42.235482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61b915-9eb9-4c2a-a429-3936a64f59df admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:42.235926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61b915-9eb9-4c2a-a429-3936a64f59df admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:42.236303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61b915-9eb9-4c2a-a429-3936a64f59df admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:42.237226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61b915-9eb9-4c2a-a429-3936a64f59df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:42.239588 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61b915-9eb9-4c2a-a429-3936a64f59df admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:42.263668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1055/2135] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:42.264473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b04f7d-9bc9-4db5-8f10-de6cb020f317 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:42.278200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b04f7d-9bc9-4db5-8f10-de6cb020f317 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:42.290799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2aceb-52b2-4106-bfc6-7e5df74eee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:42.291785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2aceb-52b2-4106-bfc6-7e5df74eee7c admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:42.292259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2aceb-52b2-4106-bfc6-7e5df74eee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:42.292714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2aceb-52b2-4106-bfc6-7e5df74eee7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:42.293590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2aceb-52b2-4106-bfc6-7e5df74eee7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:42.295382 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2aceb-52b2-4106-bfc6-7e5df74eee7c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:42.309906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1081/2136] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:42.314173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61b915-9eb9-4c2a-a429-3936a64f59df admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:42.329940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61b915-9eb9-4c2a-a429-3936a64f59df admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:42.343200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da961c03-04d9-45d8-9019-8e57644f5599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:42.344158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da961c03-04d9-45d8-9019-8e57644f5599 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:42.347327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da961c03-04d9-45d8-9019-8e57644f5599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:42.347327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da961c03-04d9-45d8-9019-8e57644f5599 admin admin] PATH_INFO: `/v3/projects/e70676ad7c38433ebc860089993f7fce/users/33075f6720d1482f9966858ca3f2d71e/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:42.372265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:42.377442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da961c03-04d9-45d8-9019-8e57644f5599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70676ad7c38433ebc860089993f7fce, user_id=33075f6720d1482f9966858ca3f2d71e, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:42.380339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da961c03-04d9-45d8-9019-8e57644f5599 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:42.507184 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1056/2137] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:42 2022] PUT /identity/v3/projects/e70676ad7c38433ebc860089993f7fce/users/33075f6720d1482f9966858ca3f2d71e/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:42.518075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104a191-064d-4d49-b3a6-8ec5e83318e9 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:42.518307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104a191-064d-4d49-b3a6-8ec5e83318e9 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:42.518577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104a191-064d-4d49-b3a6-8ec5e83318e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:42.545873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d104a191-064d-4d49-b3a6-8ec5e83318e9 None None] MFA Rules not processed for user `33075f6720d1482f9966858ca3f2d71e`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:42.547167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d104a191-064d-4d49-b3a6-8ec5e83318e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:42.652690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1082/2138] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:42.659771 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da961c03-04d9-45d8-9019-8e57644f5599 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:42.730663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da961c03-04d9-45d8-9019-8e57644f5599 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:42.737379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3038e-8959-454f-81d2-566bdca4e2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:42.738104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3038e-8959-454f-81d2-566bdca4e2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:42.738397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3038e-8959-454f-81d2-566bdca4e2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:42.738679 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3038e-8959-454f-81d2-566bdca4e2f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:42.739489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3038e-8959-454f-81d2-566bdca4e2f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:42.742077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3038e-8959-454f-81d2-566bdca4e2f9 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:42.840340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1057/2139] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:42 2022] POST /identity/v3/projects => generated 391 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:42.844519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104a191-064d-4d49-b3a6-8ec5e83318e9 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:42.855965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104a191-064d-4d49-b3a6-8ec5e83318e9 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:42.920225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d2016-8ecc-4dd3-90cf-8da3f5928d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:42.920907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d2016-8ecc-4dd3-90cf-8da3f5928d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:42.921459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d2016-8ecc-4dd3-90cf-8da3f5928d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:42.921459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d2016-8ecc-4dd3-90cf-8da3f5928d12 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:42.922264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d2016-8ecc-4dd3-90cf-8da3f5928d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:42.924309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d2016-8ecc-4dd3-90cf-8da3f5928d12 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:43.010848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1083/2140] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:42 2022] POST /identity/v3/users => generated 302 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:43.014930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3038e-8959-454f-81d2-566bdca4e2f9 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:43.026936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3038e-8959-454f-81d2-566bdca4e2f9 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:43.033652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4717e-c035-4392-84af-2adb8138bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:43.034338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4717e-c035-4392-84af-2adb8138bc50 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:43.034620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4717e-c035-4392-84af-2adb8138bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:43.035053 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4717e-c035-4392-84af-2adb8138bc50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:43.035661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4717e-c035-4392-84af-2adb8138bc50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:43.039882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4717e-c035-4392-84af-2adb8138bc50 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:43.062970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1058/2141] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:43.063296 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d2016-8ecc-4dd3-90cf-8da3f5928d12 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:43.072957 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d2016-8ecc-4dd3-90cf-8da3f5928d12 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:43.085336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f41791-0c63-4459-a471-f60c2f4248c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:43.086382 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f41791-0c63-4459-a471-f60c2f4248c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:43.086667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f41791-0c63-4459-a471-f60c2f4248c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:43.086942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f41791-0c63-4459-a471-f60c2f4248c6 admin admin] PATH_INFO: `/v3/projects/3275fe8f7af14cf9b2259e9f03c0415f/users/547aef8d6fde49febd7099356371fd1d/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:43.105459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f41791-0c63-4459-a471-f60c2f4248c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3275fe8f7af14cf9b2259e9f03c0415f, user_id=547aef8d6fde49febd7099356371fd1d, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:43.108222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f41791-0c63-4459-a471-f60c2f4248c6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:43.171271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:43.209493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1084/2142] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:43 2022] PUT /identity/v3/projects/3275fe8f7af14cf9b2259e9f03c0415f/users/547aef8d6fde49febd7099356371fd1d/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:43.212109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4717e-c035-4392-84af-2adb8138bc50 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:43.225058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4717e-c035-4392-84af-2adb8138bc50 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:43.386893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d836d-629d-4ea1-91ef-16c3f9c6c0f4 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:43.386893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d836d-629d-4ea1-91ef-16c3f9c6c0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:43.386893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d836d-629d-4ea1-91ef-16c3f9c6c0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:43.387701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b98bd-24a0-4f1c-9ce1-9bbaad3d43ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:43.387701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b98bd-24a0-4f1c-9ce1-9bbaad3d43ce admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:43.387701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b98bd-24a0-4f1c-9ce1-9bbaad3d43ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:43.387701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b98bd-24a0-4f1c-9ce1-9bbaad3d43ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:43.388400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b98bd-24a0-4f1c-9ce1-9bbaad3d43ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:43.388400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b98bd-24a0-4f1c-9ce1-9bbaad3d43ce admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:43.388400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1059/2143] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:43.388667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-184d836d-629d-4ea1-91ef-16c3f9c6c0f4 None None] MFA Rules not processed for user `cc8a087e97424d2890952be33695378f`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:43.388667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-184d836d-629d-4ea1-91ef-16c3f9c6c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:43.390406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b98bd-24a0-4f1c-9ce1-9bbaad3d43ce admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:43.417763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b98bd-24a0-4f1c-9ce1-9bbaad3d43ce admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:43.422505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125f2d1-a2d5-4c6f-b954-9b28ec032dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:43.423393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f2d1-a2d5-4c6f-b954-9b28ec032dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:43.423712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f2d1-a2d5-4c6f-b954-9b28ec032dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:43.424048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f2d1-a2d5-4c6f-b954-9b28ec032dbc admin admin] PATH_INFO: `/v3/projects/3275fe8f7af14cf9b2259e9f03c0415f/users/547aef8d6fde49febd7099356371fd1d/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:43.425957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125f2d1-a2d5-4c6f-b954-9b28ec032dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3275fe8f7af14cf9b2259e9f03c0415f, user_id=547aef8d6fde49febd7099356371fd1d, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:43.428809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125f2d1-a2d5-4c6f-b954-9b28ec032dbc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:43.450123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1060/2145] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:43 2022] PUT /identity/v3/projects/3275fe8f7af14cf9b2259e9f03c0415f/users/547aef8d6fde49febd7099356371fd1d/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:43.453765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1085/2145] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:43.453896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b33f3-5500-4cab-a4c4-f0706d737e56 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:43.453896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b33f3-5500-4cab-a4c4-f0706d737e56 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:43.453896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b33f3-5500-4cab-a4c4-f0706d737e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:43.466800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d836d-629d-4ea1-91ef-16c3f9c6c0f4 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:43.468311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-184d836d-629d-4ea1-91ef-16c3f9c6c0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:43.486841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d88b33f3-5500-4cab-a4c4-f0706d737e56 None None] MFA Rules not processed for user `547aef8d6fde49febd7099356371fd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:43.488336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d88b33f3-5500-4cab-a4c4-f0706d737e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:43.569449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1061/2146] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:43.579940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d836d-629d-4ea1-91ef-16c3f9c6c0f4 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:43.583644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b33f3-5500-4cab-a4c4-f0706d737e56 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:43.585443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d88b33f3-5500-4cab-a4c4-f0706d737e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:43.615363 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:43.667140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcae49-9b9b-4df5-a3f2-c1ebb9a752f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:43.667965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcae49-9b9b-4df5-a3f2-c1ebb9a752f9 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:43.668313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcae49-9b9b-4df5-a3f2-c1ebb9a752f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:43.668692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcae49-9b9b-4df5-a3f2-c1ebb9a752f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:43.671163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b33f3-5500-4cab-a4c4-f0706d737e56 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:43.676110 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd6b42-cf2f-4534-a7b9-d46c4adcc77b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:43.677151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd6b42-cf2f-4534-a7b9-d46c4adcc77b service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:43.677578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd6b42-cf2f-4534-a7b9-d46c4adcc77b service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:43.677994 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd6b42-cf2f-4534-a7b9-d46c4adcc77b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:43.690686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd6b42-cf2f-4534-a7b9-d46c4adcc77b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:43.692392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd6b42-cf2f-4534-a7b9-d46c4adcc77b service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:43.718558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1062/2147] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:43.763157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcae49-9b9b-4df5-a3f2-c1ebb9a752f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:43.764808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcae49-9b9b-4df5-a3f2-c1ebb9a752f9 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:43.770929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd6b42-cf2f-4534-a7b9-d46c4adcc77b service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:43.776040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8cdd6b42-cf2f-4534-a7b9-d46c4adcc77b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:43.776508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1086/2148] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:43.784859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd6b42-cf2f-4534-a7b9-d46c4adcc77b service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:43.789310 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ee0d-a8d6-4da3-b59b-2b772cdb4315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:43.790225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ee0d-a8d6-4da3-b59b-2b772cdb4315 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:43.790518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ee0d-a8d6-4da3-b59b-2b772cdb4315 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:43.790837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ee0d-a8d6-4da3-b59b-2b772cdb4315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:43.867876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6ee0d-a8d6-4da3-b59b-2b772cdb4315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:43.869368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6ee0d-a8d6-4da3-b59b-2b772cdb4315 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:43.879582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1063/2149] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:37:43.907492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:46.939417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a9c17-a80b-475f-b4f3-efe9ddeaea9f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:46.940053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a9c17-a80b-475f-b4f3-efe9ddeaea9f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:46.940134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a9c17-a80b-475f-b4f3-efe9ddeaea9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:46.968994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-cb6a9c17-a80b-475f-b4f3-efe9ddeaea9f None None] MFA Rules not processed for user `cc8a087e97424d2890952be33695378f`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:46.970371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-cb6a9c17-a80b-475f-b4f3-efe9ddeaea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:47.025150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1087/2150] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:49.217009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ee0d-a8d6-4da3-b59b-2b772cdb4315 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:49.221432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-39b6ee0d-a8d6-4da3-b59b-2b772cdb4315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:49.309225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ee0d-a8d6-4da3-b59b-2b772cdb4315 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:49.314241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:49.315142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:49.315421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:49.315855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] PATH_INFO: `/v3/users/4c2805cc63ec44fdba477745e78119b0` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:49.317060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2805cc63ec44fdba477745e78119b0)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:49.319017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:49.335508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:49.506949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2805cc63ec44fdba477745e78119b0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:49.507978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2805cc63ec44fdba477745e78119b0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:49.514626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2805cc63ec44fdba477745e78119b0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:49.515304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2805cc63ec44fdba477745e78119b0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:49.527238 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2805cc63ec44fdba477745e78119b0'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:49.622366 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1064/2151] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:49 2022] DELETE /identity/v3/users/4c2805cc63ec44fdba477745e78119b0 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:49.892140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a9c17-a80b-475f-b4f3-efe9ddeaea9f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:49.904364 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a9c17-a80b-475f-b4f3-efe9ddeaea9f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:49.970925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:49.970925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:49.971264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:49.971362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] PATH_INFO: `/v3/projects/cbb94df7abce44ddb826b8745bae1707` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:49.972575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb94df7abce44ddb826b8745bae1707)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:49.974499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:49.987839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:49.989304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:50.002977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f9d7f-af38-4089-9237-6fe4c00dfbf7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:50.014987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:50.015977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:50.016265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:50.016555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] PATH_INFO: `/v3/users/9bb2774a5cbd40598462d019fd330c3a` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:50.018287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb2774a5cbd40598462d019fd330c3a)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:50.020737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:50.114640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb94df7abce44ddb826b8745bae1707', 'request_id': 'req-36aa78c4-31d7-4173-a65c-4066e69c8a8d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.181240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb2774a5cbd40598462d019fd330c3a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.181940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb2774a5cbd40598462d019fd330c3a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.186509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb94df7abce44ddb826b8745bae1707', 'request_id': 'req-36aa78c4-31d7-4173-a65c-4066e69c8a8d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.189384 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb2774a5cbd40598462d019fd330c3a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.190050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb2774a5cbd40598462d019fd330c3a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.191178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:50.193222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb94df7abce44ddb826b8745bae1707', 'request_id': 'req-36aa78c4-31d7-4173-a65c-4066e69c8a8d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.203246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb2774a5cbd40598462d019fd330c3a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.205550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] The token cache is being invalidate because project cbb94df7abce44ddb826b8745bae1707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:50.205838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.206466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.208035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1088/2152] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:49 2022] DELETE /identity/v3/projects/cbb94df7abce44ddb826b8745bae1707 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:50.212523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:50.213810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:50.275616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa78c4-31d7-4173-a65c-4066e69c8a8d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:50.280494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:50.281360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:50.281641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:50.281913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] PATH_INFO: `/v3/users/33075f6720d1482f9966858ca3f2d71e` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:50.283332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33075f6720d1482f9966858ca3f2d71e)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:50.285772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:50.294947 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1065/2153] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:49 2022] DELETE /identity/v3/users/9bb2774a5cbd40598462d019fd330c3a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:50.296355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:50.297680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:50.369507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfffda8-741c-4304-a3bc-72a556987d69 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:50.374822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:50.375733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:50.376181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:50.376938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] PATH_INFO: `/v3/users/8fe955dec19741bc89ed032422ad3ed5` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:50.378010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fe955dec19741bc89ed032422ad3ed5)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:50.380245 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:50.387034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:50.451375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33075f6720d1482f9966858ca3f2d71e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.523905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33075f6720d1482f9966858ca3f2d71e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.524538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33075f6720d1482f9966858ca3f2d71e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.532557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33075f6720d1482f9966858ca3f2d71e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.532557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4aeba67d-89ca-492e-88e1-f58c1fc26585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33075f6720d1482f9966858ca3f2d71e'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.549700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1089/2154] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:50 2022] DELETE /identity/v3/users/33075f6720d1482f9966858ca3f2d71e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:50.551163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04615c0f-e28a-40e0-aa57-c6dcec408633 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:50.551383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04615c0f-e28a-40e0-aa57-c6dcec408633 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:50.551713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04615c0f-e28a-40e0-aa57-c6dcec408633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:50.566886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fe955dec19741bc89ed032422ad3ed5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.567732 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fe955dec19741bc89ed032422ad3ed5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.568791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-04615c0f-e28a-40e0-aa57-c6dcec408633 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:50.570637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-04615c0f-e28a-40e0-aa57-c6dcec408633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:50.575290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fe955dec19741bc89ed032422ad3ed5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.575967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fe955dec19741bc89ed032422ad3ed5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.588054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fe955dec19741bc89ed032422ad3ed5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:50.629278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1090/2155] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:50.630573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b80e6-99ce-4042-ad11-e615cc3f0ecd None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:50.630809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b80e6-99ce-4042-ad11-e615cc3f0ecd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:50.631110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b80e6-99ce-4042-ad11-e615cc3f0ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:50.649348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-003b80e6-99ce-4042-ad11-e615cc3f0ecd None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:50.650556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-003b80e6-99ce-4042-ad11-e615cc3f0ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:50.658255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1091/2156] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:50.659010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b80e6-99ce-4042-ad11-e615cc3f0ecd None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:50.660358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-003b80e6-99ce-4042-ad11-e615cc3f0ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:50.665653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:50.668651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1066/2157] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:50 2022] DELETE /identity/v3/users/8fe955dec19741bc89ed032422ad3ed5 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:50.669687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:50.671086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:50.727025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b80e6-99ce-4042-ad11-e615cc3f0ecd None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:50.732596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2a830-1261-46f4-a667-ce55c1aa51ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:50.733440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2a830-1261-46f4-a667-ce55c1aa51ae service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:50.734152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2a830-1261-46f4-a667-ce55c1aa51ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:50.734606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2a830-1261-46f4-a667-ce55c1aa51ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:50.736020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9251b-c969-4b85-a7d9-c82e7ed49fd0 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:50.740217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870a022-de6a-4d82-bfe9-3e721085d697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:50.741388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870a022-de6a-4d82-bfe9-3e721085d697 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:50.741695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870a022-de6a-4d82-bfe9-3e721085d697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:50.741957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870a022-de6a-4d82-bfe9-3e721085d697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:50.805228 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2a830-1261-46f4-a667-ce55c1aa51ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:50.806682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2a830-1261-46f4-a667-ce55c1aa51ae service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:50.812145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870a022-de6a-4d82-bfe9-3e721085d697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:50.813221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870a022-de6a-4d82-bfe9-3e721085d697 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:50.817025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1092/2158] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:50.824613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1067/2159] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:50.945200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2a830-1261-46f4-a667-ce55c1aa51ae service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:50.945200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c1b2a830-1261-46f4-a667-ce55c1aa51ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:50.958150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2a830-1261-46f4-a667-ce55c1aa51ae service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:50.963872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:50.965022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:50.965397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:50.965774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] PATH_INFO: `/v3/projects/c6581f8a6ea74ab7b7e89f09d97eebfe` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:50.967501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6581f8a6ea74ab7b7e89f09d97eebfe)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:50.969515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:51.117467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6581f8a6ea74ab7b7e89f09d97eebfe', 'request_id': 'req-69f3f566-4721-417a-8827-e8f02700a3ae'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.198245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6581f8a6ea74ab7b7e89f09d97eebfe', 'request_id': 'req-69f3f566-4721-417a-8827-e8f02700a3ae'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.204830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6581f8a6ea74ab7b7e89f09d97eebfe', 'request_id': 'req-69f3f566-4721-417a-8827-e8f02700a3ae'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.226442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] The token cache is being invalidate because project c6581f8a6ea74ab7b7e89f09d97eebfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:51.226733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.227403 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69f3f566-4721-417a-8827-e8f02700a3ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.231974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1093/2160] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:50 2022] DELETE /identity/v3/projects/c6581f8a6ea74ab7b7e89f09d97eebfe => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:51.237205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68f776-e663-4d14-ac02-8e63a06fc4a4 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.237494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68f776-e663-4d14-ac02-8e63a06fc4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.237769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68f776-e663-4d14-ac02-8e63a06fc4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.259853 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ad68f776-e663-4d14-ac02-8e63a06fc4a4 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:51.261824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ad68f776-e663-4d14-ac02-8e63a06fc4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:51.320382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:51.351483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1068/2161] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:51.360216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1b7c5-ad3f-4e94-8e9c-e97f8fb0649b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.363130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1b7c5-ad3f-4e94-8e9c-e97f8fb0649b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.363130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1b7c5-ad3f-4e94-8e9c-e97f8fb0649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.366666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68f776-e663-4d14-ac02-8e63a06fc4a4 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:51.370346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ad68f776-e663-4d14-ac02-8e63a06fc4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:51.383985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68f776-e663-4d14-ac02-8e63a06fc4a4 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:51.388341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-eea1b7c5-ad3f-4e94-8e9c-e97f8fb0649b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:51.390681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-eea1b7c5-ad3f-4e94-8e9c-e97f8fb0649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:51.390794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:51.391373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.391930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.392213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] PATH_INFO: `/v3/projects/c00a0cb2e37c4689ace91bc2a1086166` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.393436 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=c00a0cb2e37c4689ace91bc2a1086166)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:51.395406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:51.401271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1094/2162] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:51.406526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1b7c5-ad3f-4e94-8e9c-e97f8fb0649b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:51.439675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1b7c5-ad3f-4e94-8e9c-e97f8fb0649b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:51.446260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16038ffc-2004-4c7b-abc6-a587cb3219a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:51.447050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16038ffc-2004-4c7b-abc6-a587cb3219a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.447337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16038ffc-2004-4c7b-abc6-a587cb3219a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.447659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16038ffc-2004-4c7b-abc6-a587cb3219a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.448401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16038ffc-2004-4c7b-abc6-a587cb3219a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:51.448766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16038ffc-2004-4c7b-abc6-a587cb3219a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:51.450654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16038ffc-2004-4c7b-abc6-a587cb3219a1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:51.471725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1095/2163] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:37:51.473719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16038ffc-2004-4c7b-abc6-a587cb3219a1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:51.484671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16038ffc-2004-4c7b-abc6-a587cb3219a1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:51.489816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c729f00-5b82-48fb-b0c3-18af6fbcbcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:51.493290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c729f00-5b82-48fb-b0c3-18af6fbcbcd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.493290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c729f00-5b82-48fb-b0c3-18af6fbcbcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.493290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c729f00-5b82-48fb-b0c3-18af6fbcbcd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.493290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c729f00-5b82-48fb-b0c3-18af6fbcbcd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:51.493635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c729f00-5b82-48fb-b0c3-18af6fbcbcd9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:51.507975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:51.564319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c00a0cb2e37c4689ace91bc2a1086166', 'request_id': 'req-ca0edf9d-a752-4124-8e16-e76734b08dda'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.569653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c00a0cb2e37c4689ace91bc2a1086166', 'request_id': 'req-ca0edf9d-a752-4124-8e16-e76734b08dda'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.575831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c00a0cb2e37c4689ace91bc2a1086166', 'request_id': 'req-ca0edf9d-a752-4124-8e16-e76734b08dda'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.582335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1096/2164] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:51 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:51.583194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c729f00-5b82-48fb-b0c3-18af6fbcbcd9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:51.587454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9c729f00-5b82-48fb-b0c3-18af6fbcbcd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:51.650120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c729f00-5b82-48fb-b0c3-18af6fbcbcd9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:51.654930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:51.655823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.656112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.656393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] PATH_INFO: `/v3/projects/e70676ad7c38433ebc860089993f7fce` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.657593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70676ad7c38433ebc860089993f7fce)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:51.659514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:51.660514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] The token cache is being invalidate because project c00a0cb2e37c4689ace91bc2a1086166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:51.660841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.661480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.666429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1069/2165] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:51 2022] DELETE /identity/v3/projects/c00a0cb2e37c4689ace91bc2a1086166 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:51.666429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:51.666429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:51.728502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0edf9d-a752-4124-8e16-e76734b08dda admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:51.733215 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d04e7-6a89-424f-ab99-9591329d1e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:51.733936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d04e7-6a89-424f-ab99-9591329d1e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.734224 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d04e7-6a89-424f-ab99-9591329d1e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.734505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d04e7-6a89-424f-ab99-9591329d1e67 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.735202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d04e7-6a89-424f-ab99-9591329d1e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:51.737162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d04e7-6a89-424f-ab99-9591329d1e67 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:51.758819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70676ad7c38433ebc860089993f7fce', 'request_id': 'req-69e35c23-3b16-45d5-a2f2-291e40704246'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.838612 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1070/2166] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:51 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:51.839365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d04e7-6a89-424f-ab99-9591329d1e67 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:51.840954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8e2d04e7-6a89-424f-ab99-9591329d1e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:51.844133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70676ad7c38433ebc860089993f7fce', 'request_id': 'req-69e35c23-3b16-45d5-a2f2-291e40704246'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.849924 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70676ad7c38433ebc860089993f7fce', 'request_id': 'req-69e35c23-3b16-45d5-a2f2-291e40704246'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.853002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d04e7-6a89-424f-ab99-9591329d1e67 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:51.861002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] The token cache is being invalidate because project e70676ad7c38433ebc860089993f7fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:51.861356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.862008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:51.863838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1097/2167] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:51 2022] DELETE /identity/v3/projects/e70676ad7c38433ebc860089993f7fce => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:51.864336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:51.865619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:51.911338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:51.912318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.912613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.913026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] PATH_INFO: `/v3/users/90c3fbee94f14235940660245c144518` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.914255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c3fbee94f14235940660245c144518)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:51.916536 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:51.929229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e35c23-3b16-45d5-a2f2-291e40704246 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:51.933951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a252e5b-d05c-4a3a-9a16-df8dfa57e3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:51.934608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a252e5b-d05c-4a3a-9a16-df8dfa57e3cb admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.934888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a252e5b-d05c-4a3a-9a16-df8dfa57e3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.935167 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a252e5b-d05c-4a3a-9a16-df8dfa57e3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.935922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a252e5b-d05c-4a3a-9a16-df8dfa57e3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:51.937504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a252e5b-d05c-4a3a-9a16-df8dfa57e3cb admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:51.939185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:51.947893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1098/2168] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:51.948718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a252e5b-d05c-4a3a-9a16-df8dfa57e3cb admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:51.953222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5a252e5b-d05c-4a3a-9a16-df8dfa57e3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:51.963361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a252e5b-d05c-4a3a-9a16-df8dfa57e3cb admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:51.969258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:51.970086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:51.970368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:51.970642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] PATH_INFO: `/v3/users/547aef8d6fde49febd7099356371fd1d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:51.971890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] RBAC: Authorizing `identity:delete_user(user_id=547aef8d6fde49febd7099356371fd1d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:51.973824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:51.987255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:52.088537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c3fbee94f14235940660245c144518'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.089264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c3fbee94f14235940660245c144518'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.096229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c3fbee94f14235940660245c144518'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.097492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c3fbee94f14235940660245c144518'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.108728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c3fbee94f14235940660245c144518'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.131408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547aef8d6fde49febd7099356371fd1d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.209781 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1071/2169] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:51 2022] DELETE /identity/v3/users/90c3fbee94f14235940660245c144518 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:52.210693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:52.212057 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:52.276238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547aef8d6fde49febd7099356371fd1d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.276880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547aef8d6fde49febd7099356371fd1d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.282914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f8298-b86d-458a-acb7-cb78565365fa admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:52.291727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0a55d-0acf-4bfc-9a9a-8f064b520006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:52.296924 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547aef8d6fde49febd7099356371fd1d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.297772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547aef8d6fde49febd7099356371fd1d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.298108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0a55d-0acf-4bfc-9a9a-8f064b520006 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:52.298847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0a55d-0acf-4bfc-9a9a-8f064b520006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:52.299651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0a55d-0acf-4bfc-9a9a-8f064b520006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:52.301785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0a55d-0acf-4bfc-9a9a-8f064b520006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:52.304853 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0a55d-0acf-4bfc-9a9a-8f064b520006 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:52.317526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1099/2170] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:51 2022] DELETE /identity/v3/users/547aef8d6fde49febd7099356371fd1d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:52.321489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1072/2171] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:52.322542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:52.323907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:52.395523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78284d-d0c7-49d0-a797-c4b60a98866f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:52.401419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada8453-40fd-4296-8844-0055a76ef7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:52.402394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8453-40fd-4296-8844-0055a76ef7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:52.402678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8453-40fd-4296-8844-0055a76ef7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:52.402954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8453-40fd-4296-8844-0055a76ef7e7 admin admin] PATH_INFO: `/v3/projects/d6a8fa7f6bdb43c4941aeb6a22a6c1bc/users/eb462af0f24c4ddebe605d8ed636b8ea/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:52.419710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada8453-40fd-4296-8844-0055a76ef7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a8fa7f6bdb43c4941aeb6a22a6c1bc, user_id=eb462af0f24c4ddebe605d8ed636b8ea, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:52.422421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada8453-40fd-4296-8844-0055a76ef7e7 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:52.534623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1100/2172] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:52 2022] PUT /identity/v3/projects/d6a8fa7f6bdb43c4941aeb6a22a6c1bc/users/eb462af0f24c4ddebe605d8ed636b8ea/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:52.536433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed181c84-9b21-4f6f-8345-1fc9b2f27256 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:52.537010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed181c84-9b21-4f6f-8345-1fc9b2f27256 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:52.537505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed181c84-9b21-4f6f-8345-1fc9b2f27256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:52.588586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ed181c84-9b21-4f6f-8345-1fc9b2f27256 None None] MFA Rules not processed for user `eb462af0f24c4ddebe605d8ed636b8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:52.590830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ed181c84-9b21-4f6f-8345-1fc9b2f27256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:52.673978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:52.675650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada8453-40fd-4296-8844-0055a76ef7e7 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:52.678126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fada8453-40fd-4296-8844-0055a76ef7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:52.695916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1073/2173] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:52.710490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed181c84-9b21-4f6f-8345-1fc9b2f27256 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:52.712343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ed181c84-9b21-4f6f-8345-1fc9b2f27256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:52.781450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada8453-40fd-4296-8844-0055a76ef7e7 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:52.786444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:52.787314 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:52.787655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:52.787965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] PATH_INFO: `/v3/projects/f307f719ef0e454f9b4c40ea9a57dae4` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:52.789188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f307f719ef0e454f9b4c40ea9a57dae4)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:52.791473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:52.793199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed181c84-9b21-4f6f-8345-1fc9b2f27256 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:52.797421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec64bb-268f-4b9b-9006-97bd9ca77390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:52.798158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec64bb-268f-4b9b-9006-97bd9ca77390 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:52.798450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec64bb-268f-4b9b-9006-97bd9ca77390 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:52.798729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec64bb-268f-4b9b-9006-97bd9ca77390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:52.808842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec64bb-268f-4b9b-9006-97bd9ca77390 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:52.810248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:52.810436 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec64bb-268f-4b9b-9006-97bd9ca77390 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:52.822406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1074/2174] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:52.895402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f307f719ef0e454f9b4c40ea9a57dae4', 'request_id': 'req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.897432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec64bb-268f-4b9b-9006-97bd9ca77390 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:52.898493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-bfec64bb-268f-4b9b-9006-97bd9ca77390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:52.914754 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec64bb-268f-4b9b-9006-97bd9ca77390 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:52.969540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f307f719ef0e454f9b4c40ea9a57dae4', 'request_id': 'req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.976986 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:52.977230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f307f719ef0e454f9b4c40ea9a57dae4', 'request_id': 'req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.977893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:52.978175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:52.978447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] PATH_INFO: `/v3/projects/3275fe8f7af14cf9b2259e9f03c0415f` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:52.979810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3275fe8f7af14cf9b2259e9f03c0415f)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:52.981720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:52.990665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] The token cache is being invalidate because project f307f719ef0e454f9b4c40ea9a57dae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:52.991594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.992391 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2ac6e72e-93e5-4a39-98bb-d86d23956ea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:52.993946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1101/2175] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:52 2022] DELETE /identity/v3/projects/f307f719ef0e454f9b4c40ea9a57dae4 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:52.998628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ad43e-6a8a-4998-aa4d-9a014f8d9ad7 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:52.998916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ad43e-6a8a-4998-aa4d-9a014f8d9ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:52.999194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ad43e-6a8a-4998-aa4d-9a014f8d9ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.017605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-960ad43e-6a8a-4998-aa4d-9a014f8d9ad7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:53.018816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-960ad43e-6a8a-4998-aa4d-9a014f8d9ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:53.027000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1102/2176] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:53.034668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc5d69-719b-4b38-92d3-9822d4824a5d None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.034973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc5d69-719b-4b38-92d3-9822d4824a5d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.035246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc5d69-719b-4b38-92d3-9822d4824a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.052981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-38dc5d69-719b-4b38-92d3-9822d4824a5d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:53.054724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-38dc5d69-719b-4b38-92d3-9822d4824a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:53.063576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1103/2177] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:53.067547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc5d69-719b-4b38-92d3-9822d4824a5d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.077097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc5d69-719b-4b38-92d3-9822d4824a5d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.081743 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b3041-4326-4c15-bb3a-c06cf25acea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.082364 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b3041-4326-4c15-bb3a-c06cf25acea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.082655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b3041-4326-4c15-bb3a-c06cf25acea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.082933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b3041-4326-4c15-bb3a-c06cf25acea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.083712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b3041-4326-4c15-bb3a-c06cf25acea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:53.084027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b3041-4326-4c15-bb3a-c06cf25acea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.085635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b3041-4326-4c15-bb3a-c06cf25acea1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.101060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1104/2178] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:37:53.101925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b3041-4326-4c15-bb3a-c06cf25acea1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.109562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3275fe8f7af14cf9b2259e9f03c0415f', 'request_id': 'req-459dd447-87da-47a7-8ab3-faabd5e87302'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:53.111432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b3041-4326-4c15-bb3a-c06cf25acea1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.118508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3275fe8f7af14cf9b2259e9f03c0415f', 'request_id': 'req-459dd447-87da-47a7-8ab3-faabd5e87302'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:53.126332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3275fe8f7af14cf9b2259e9f03c0415f', 'request_id': 'req-459dd447-87da-47a7-8ab3-faabd5e87302'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:53.169517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f44c08-9e74-4f21-ba93-1ec28bbec2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.170127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44c08-9e74-4f21-ba93-1ec28bbec2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.170413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44c08-9e74-4f21-ba93-1ec28bbec2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.170690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44c08-9e74-4f21-ba93-1ec28bbec2f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.171380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f44c08-9e74-4f21-ba93-1ec28bbec2f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.173331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f44c08-9e74-4f21-ba93-1ec28bbec2f2 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.220848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] The token cache is being invalidate because project 3275fe8f7af14cf9b2259e9f03c0415f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:53.221141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:53.221821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-459dd447-87da-47a7-8ab3-faabd5e87302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:53.223429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1075/2179] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:52 2022] DELETE /identity/v3/projects/3275fe8f7af14cf9b2259e9f03c0415f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:53.229852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498890-9581-471e-b928-d944ae08bf89 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.230677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498890-9581-471e-b928-d944ae08bf89 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.230973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498890-9581-471e-b928-d944ae08bf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.252331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8a498890-9581-471e-b928-d944ae08bf89 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:53.253526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8a498890-9581-471e-b928-d944ae08bf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:53.261454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1076/2180] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:53.270076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34e40c-5e35-486c-8e04-1192e3b1596e None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.270359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34e40c-5e35-486c-8e04-1192e3b1596e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.270661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34e40c-5e35-486c-8e04-1192e3b1596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.291106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ba34e40c-5e35-486c-8e04-1192e3b1596e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:53.291106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ba34e40c-5e35-486c-8e04-1192e3b1596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:53.299729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1077/2181] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:53.304352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34e40c-5e35-486c-8e04-1192e3b1596e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.313566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34e40c-5e35-486c-8e04-1192e3b1596e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.316022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1105/2182] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:53 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:53.321161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f44c08-9e74-4f21-ba93-1ec28bbec2f2 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.323195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-97f44c08-9e74-4f21-ba93-1ec28bbec2f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:53.325276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:53.376847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113b19f-801e-4529-9163-5dcd34970fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.377547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113b19f-801e-4529-9163-5dcd34970fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.377826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113b19f-801e-4529-9163-5dcd34970fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.378123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113b19f-801e-4529-9163-5dcd34970fe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.378997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113b19f-801e-4529-9163-5dcd34970fe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:53.379607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113b19f-801e-4529-9163-5dcd34970fe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.393928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113b19f-801e-4529-9163-5dcd34970fe5 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.396893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f44c08-9e74-4f21-ba93-1ec28bbec2f2 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.401864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76445d2d-422a-490c-8bc4-1f72248d8d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.402568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76445d2d-422a-490c-8bc4-1f72248d8d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.402816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76445d2d-422a-490c-8bc4-1f72248d8d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.403093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76445d2d-422a-490c-8bc4-1f72248d8d90 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.403989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76445d2d-422a-490c-8bc4-1f72248d8d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.405986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76445d2d-422a-490c-8bc4-1f72248d8d90 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.411808 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1078/2183] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:37:53.419260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:53.421957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113b19f-801e-4529-9163-5dcd34970fe5 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.432850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113b19f-801e-4529-9163-5dcd34970fe5 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.437905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38130502-fce7-4144-8a7c-2bbdc76219c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.438634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38130502-fce7-4144-8a7c-2bbdc76219c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.439011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38130502-fce7-4144-8a7c-2bbdc76219c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.439365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38130502-fce7-4144-8a7c-2bbdc76219c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.440175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38130502-fce7-4144-8a7c-2bbdc76219c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.442303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38130502-fce7-4144-8a7c-2bbdc76219c7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.494168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1106/2184] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:53 2022] POST /identity/v3/users => generated 283 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:53.498133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76445d2d-422a-490c-8bc4-1f72248d8d90 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.508872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76445d2d-422a-490c-8bc4-1f72248d8d90 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.514241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa916-ae56-4eb2-a0d3-0f4c4ebb995d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.514926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa916-ae56-4eb2-a0d3-0f4c4ebb995d admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.518295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa916-ae56-4eb2-a0d3-0f4c4ebb995d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.518295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afa916-ae56-4eb2-a0d3-0f4c4ebb995d admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.518295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afa916-ae56-4eb2-a0d3-0f4c4ebb995d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.518295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afa916-ae56-4eb2-a0d3-0f4c4ebb995d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.528048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1079/2185] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:53 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:53.530299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38130502-fce7-4144-8a7c-2bbdc76219c7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.536432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1107/2186] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:53.544289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa916-ae56-4eb2-a0d3-0f4c4ebb995d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.577075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afa916-ae56-4eb2-a0d3-0f4c4ebb995d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.637846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38130502-fce7-4144-8a7c-2bbdc76219c7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.642676 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76276600-1ba9-4b95-8501-28cddd71691d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.643459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76276600-1ba9-4b95-8501-28cddd71691d admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.643865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76276600-1ba9-4b95-8501-28cddd71691d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.644250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76276600-1ba9-4b95-8501-28cddd71691d admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.645152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76276600-1ba9-4b95-8501-28cddd71691d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.647490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76276600-1ba9-4b95-8501-28cddd71691d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.670278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655054c-601f-4903-acfd-fc4359777dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.670278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655054c-601f-4903-acfd-fc4359777dae admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.670278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655054c-601f-4903-acfd-fc4359777dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.670278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655054c-601f-4903-acfd-fc4359777dae admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.670697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655054c-601f-4903-acfd-fc4359777dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.671845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655054c-601f-4903-acfd-fc4359777dae admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.684712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1108/2187] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:53.690768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655054c-601f-4903-acfd-fc4359777dae admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.704647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655054c-601f-4903-acfd-fc4359777dae admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.710611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c9b04-02b2-4b3f-9158-9d650c2b6435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.711773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c9b04-02b2-4b3f-9158-9d650c2b6435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.712058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c9b04-02b2-4b3f-9158-9d650c2b6435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.712335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c9b04-02b2-4b3f-9158-9d650c2b6435 admin admin] PATH_INFO: `/v3/projects/938847a2e4de4e5e9721172d81121879/users/3f47bf45459b4c4ea270900ce87ac38b/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.731195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c9b04-02b2-4b3f-9158-9d650c2b6435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=938847a2e4de4e5e9721172d81121879, user_id=3f47bf45459b4c4ea270900ce87ac38b, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.734007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c9b04-02b2-4b3f-9158-9d650c2b6435 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.760691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1080/2188] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:53 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:53.765291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76276600-1ba9-4b95-8501-28cddd71691d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.777719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76276600-1ba9-4b95-8501-28cddd71691d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.782840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5572-bda5-4000-8496-52e2719b35ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.783616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5572-bda5-4000-8496-52e2719b35ec admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.784009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5572-bda5-4000-8496-52e2719b35ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.784357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5572-bda5-4000-8496-52e2719b35ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.785308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f5572-bda5-4000-8496-52e2719b35ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.787048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f5572-bda5-4000-8496-52e2719b35ec admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.800447 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1081/2189] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:53.805733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5572-bda5-4000-8496-52e2719b35ec admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.829618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1109/2190] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:53 2022] PUT /identity/v3/projects/938847a2e4de4e5e9721172d81121879/users/3f47bf45459b4c4ea270900ce87ac38b/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:53.830863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5572-bda5-4000-8496-52e2719b35ec admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.841650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a5f90-d4bb-469d-9fb3-3d2035db81c8 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.841965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a5f90-d4bb-469d-9fb3-3d2035db81c8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.842221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a5f90-d4bb-469d-9fb3-3d2035db81c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.890398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4f2a5f90-d4bb-469d-9fb3-3d2035db81c8 None None] MFA Rules not processed for user `3f47bf45459b4c4ea270900ce87ac38b`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:53.893838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4f2a5f90-d4bb-469d-9fb3-3d2035db81c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:53.941899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9aa32-2af9-4a93-bdd2-4989068d483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.942499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9aa32-2af9-4a93-bdd2-4989068d483c admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.942793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9aa32-2af9-4a93-bdd2-4989068d483c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.943077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9aa32-2af9-4a93-bdd2-4989068d483c admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:53.944370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9aa32-2af9-4a93-bdd2-4989068d483c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:53.946430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9aa32-2af9-4a93-bdd2-4989068d483c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:53.952450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:53.958167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1082/2191] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:53.962649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9aa32-2af9-4a93-bdd2-4989068d483c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:53.973211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9aa32-2af9-4a93-bdd2-4989068d483c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:53.978402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22eb228-9299-405d-8fa0-fc5a47e8e0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:53.979302 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22eb228-9299-405d-8fa0-fc5a47e8e0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:53.979758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22eb228-9299-405d-8fa0-fc5a47e8e0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:53.980088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22eb228-9299-405d-8fa0-fc5a47e8e0b9 admin admin] PATH_INFO: `/v3/projects/f86c88c79d5f4617981ee836b5d30d2b/users/6aa05fccdbcd4ae68f5039b70037c847/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.000083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22eb228-9299-405d-8fa0-fc5a47e8e0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86c88c79d5f4617981ee836b5d30d2b, user_id=6aa05fccdbcd4ae68f5039b70037c847, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.000083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22eb228-9299-405d-8fa0-fc5a47e8e0b9 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.004254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1110/2192] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:54.009065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a5f90-d4bb-469d-9fb3-3d2035db81c8 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.019543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a5f90-d4bb-469d-9fb3-3d2035db81c8 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.024940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e8b30-59f9-45a2-ab61-aa5656c846e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.025595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e8b30-59f9-45a2-ab61-aa5656c846e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.025872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e8b30-59f9-45a2-ab61-aa5656c846e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.026147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e8b30-59f9-45a2-ab61-aa5656c846e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.026836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e8b30-59f9-45a2-ab61-aa5656c846e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.028819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e8b30-59f9-45a2-ab61-aa5656c846e1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.096031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1083/2193] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:53 2022] PUT /identity/v3/projects/f86c88c79d5f4617981ee836b5d30d2b/users/6aa05fccdbcd4ae68f5039b70037c847/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:54.106488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1a77d-bdd8-4517-964b-f68bc2a19bc6 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.106777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1a77d-bdd8-4517-964b-f68bc2a19bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.107052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1a77d-bdd8-4517-964b-f68bc2a19bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.125055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1111/2194] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:54 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:54.129277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e8b30-59f9-45a2-ab61-aa5656c846e1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.140082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-12a1a77d-bdd8-4517-964b-f68bc2a19bc6 None None] MFA Rules not processed for user `6aa05fccdbcd4ae68f5039b70037c847`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:54.141491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-12a1a77d-bdd8-4517-964b-f68bc2a19bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:54.165921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:54.207044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e8b30-59f9-45a2-ab61-aa5656c846e1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.212777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994f2a0-3d64-4975-a446-41893e5a9b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.213481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994f2a0-3d64-4975-a446-41893e5a9b52 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.213951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994f2a0-3d64-4975-a446-41893e5a9b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.214236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994f2a0-3d64-4975-a446-41893e5a9b52 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.214928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994f2a0-3d64-4975-a446-41893e5a9b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.217104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994f2a0-3d64-4975-a446-41893e5a9b52 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.218118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1084/2195] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:54.229786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1a77d-bdd8-4517-964b-f68bc2a19bc6 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.231523 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-12a1a77d-bdd8-4517-964b-f68bc2a19bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:54.293123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1a77d-bdd8-4517-964b-f68bc2a19bc6 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.297433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f45f85-6be6-4788-a66e-fab9e0131fbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.298176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f45f85-6be6-4788-a66e-fab9e0131fbc service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.298482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f45f85-6be6-4788-a66e-fab9e0131fbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.298759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f45f85-6be6-4788-a66e-fab9e0131fbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.308264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f45f85-6be6-4788-a66e-fab9e0131fbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.312074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f45f85-6be6-4788-a66e-fab9e0131fbc service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.319794 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1085/2196] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:37:54.339634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1112/2197] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:54 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:54.343320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f45f85-6be6-4788-a66e-fab9e0131fbc service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.346486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994f2a0-3d64-4975-a446-41893e5a9b52 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.360542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994f2a0-3d64-4975-a446-41893e5a9b52 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.367193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.368102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.368304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f45f85-6be6-4788-a66e-fab9e0131fbc service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.368506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.368882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] PATH_INFO: `/v3/users/6aa05fccdbcd4ae68f5039b70037c847` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.370222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa05fccdbcd4ae68f5039b70037c847)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.372657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.378397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35f99e-30af-4121-a0de-6241f6b5d091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.379004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35f99e-30af-4121-a0de-6241f6b5d091 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.379391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35f99e-30af-4121-a0de-6241f6b5d091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.379895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35f99e-30af-4121-a0de-6241f6b5d091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.383095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35f99e-30af-4121-a0de-6241f6b5d091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.386075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35f99e-30af-4121-a0de-6241f6b5d091 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.404565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1086/2198] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:54.405457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35f99e-30af-4121-a0de-6241f6b5d091 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.417977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35f99e-30af-4121-a0de-6241f6b5d091 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.422779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a17896-418a-44f7-9bdf-ce93c0b33ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.423698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a17896-418a-44f7-9bdf-ce93c0b33ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.423983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a17896-418a-44f7-9bdf-ce93c0b33ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.424265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a17896-418a-44f7-9bdf-ce93c0b33ef3 admin admin] PATH_INFO: `/v3/projects/c53560294e954b1888c489d4ae243fdb/users/9b3fdd32caed47d98027c658408bf2c5/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.439982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a17896-418a-44f7-9bdf-ce93c0b33ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53560294e954b1888c489d4ae243fdb, user_id=9b3fdd32caed47d98027c658408bf2c5, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.442717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a17896-418a-44f7-9bdf-ce93c0b33ef3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.538725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1087/2199] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:54 2022] PUT /identity/v3/projects/c53560294e954b1888c489d4ae243fdb/users/9b3fdd32caed47d98027c658408bf2c5/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:54.540166 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a17896-418a-44f7-9bdf-ce93c0b33ef3 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.585997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:54.604622 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa05fccdbcd4ae68f5039b70037c847'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:54.622374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a17896-418a-44f7-9bdf-ce93c0b33ef3 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.628115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0297f5-fe1d-4933-93d6-02be23b3d7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.628999 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0297f5-fe1d-4933-93d6-02be23b3d7de admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.629286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0297f5-fe1d-4933-93d6-02be23b3d7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.629567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0297f5-fe1d-4933-93d6-02be23b3d7de admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.630309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0297f5-fe1d-4933-93d6-02be23b3d7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.632730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0297f5-fe1d-4933-93d6-02be23b3d7de admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.645021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1088/2200] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:54.649437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0297f5-fe1d-4933-93d6-02be23b3d7de admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.661563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0297f5-fe1d-4933-93d6-02be23b3d7de admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.667618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719eb8f-1c98-41d4-9b20-c96b4e988e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.668653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719eb8f-1c98-41d4-9b20-c96b4e988e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.668950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719eb8f-1c98-41d4-9b20-c96b4e988e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.669231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719eb8f-1c98-41d4-9b20-c96b4e988e46 admin admin] PATH_INFO: `/v3/projects/c53560294e954b1888c489d4ae243fdb/users/9b3fdd32caed47d98027c658408bf2c5/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.672180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719eb8f-1c98-41d4-9b20-c96b4e988e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53560294e954b1888c489d4ae243fdb, user_id=9b3fdd32caed47d98027c658408bf2c5, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.673533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa05fccdbcd4ae68f5039b70037c847'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:54.674214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa05fccdbcd4ae68f5039b70037c847'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:54.684637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa05fccdbcd4ae68f5039b70037c847'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:54.685656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c73ffac8-2d2e-447b-b393-3981acb7f49a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa05fccdbcd4ae68f5039b70037c847'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:54.688762 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719eb8f-1c98-41d4-9b20-c96b4e988e46 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.716003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1089/2201] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:54 2022] PUT /identity/v3/projects/c53560294e954b1888c489d4ae243fdb/users/9b3fdd32caed47d98027c658408bf2c5/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:54.722691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1113/2202] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:54 2022] DELETE /identity/v3/users/6aa05fccdbcd4ae68f5039b70037c847 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:54.725841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817d334-df82-46df-a3ed-0c9d8baf616d None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.726138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817d334-df82-46df-a3ed-0c9d8baf616d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.726479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817d334-df82-46df-a3ed-0c9d8baf616d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.730688 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb79af3-a177-4ba6-94dc-e7b021fa12ab None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.731961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb79af3-a177-4ba6-94dc-e7b021fa12ab None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.732249 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb79af3-a177-4ba6-94dc-e7b021fa12ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.752322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e817d334-df82-46df-a3ed-0c9d8baf616d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:54.753560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e817d334-df82-46df-a3ed-0c9d8baf616d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:54.776528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-6bb79af3-a177-4ba6-94dc-e7b021fa12ab None None] MFA Rules not processed for user `9b3fdd32caed47d98027c658408bf2c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:54.778030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6bb79af3-a177-4ba6-94dc-e7b021fa12ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:54.859051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1090/2203] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:54.860407 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1114/2204] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:54 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:54.870696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817d334-df82-46df-a3ed-0c9d8baf616d None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.873303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e817d334-df82-46df-a3ed-0c9d8baf616d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:54.882979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb79af3-a177-4ba6-94dc-e7b021fa12ab None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:54.884788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6bb79af3-a177-4ba6-94dc-e7b021fa12ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:54.898479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:54.950239 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817d334-df82-46df-a3ed-0c9d8baf616d None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.955382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc754a-476e-46b4-b330-c72ab24dd32b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.956532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc754a-476e-46b4-b330-c72ab24dd32b service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.957002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc754a-476e-46b4-b330-c72ab24dd32b service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.957283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc754a-476e-46b4-b330-c72ab24dd32b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.966715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb79af3-a177-4ba6-94dc-e7b021fa12ab None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:54.966715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ba4d-ac9b-4733-bb07-0f3d4abfac9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:54.966715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ba4d-ac9b-4733-bb07-0f3d4abfac9c service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:54.966715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ba4d-ac9b-4733-bb07-0f3d4abfac9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:54.967135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ba4d-ac9b-4733-bb07-0f3d4abfac9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:54.968587 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc754a-476e-46b4-b330-c72ab24dd32b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.970171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc754a-476e-46b4-b330-c72ab24dd32b service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.977448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281ba4d-ac9b-4733-bb07-0f3d4abfac9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:54.979053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281ba4d-ac9b-4733-bb07-0f3d4abfac9c service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:54.987370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1091/2205] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:54.989159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1115/2206] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:55.623563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc754a-476e-46b4-b330-c72ab24dd32b service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:55.625087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-58fc754a-476e-46b4-b330-c72ab24dd32b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:55.659899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc754a-476e-46b4-b330-c72ab24dd32b service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:55.679244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:55.683703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:55.683703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:55.683703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] PATH_INFO: `/v3/projects/f86c88c79d5f4617981ee836b5d30d2b` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:55.684563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86c88c79d5f4617981ee836b5d30d2b)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:55.686004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:55.821812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86c88c79d5f4617981ee836b5d30d2b', 'request_id': 'req-9b7e0824-8a24-4904-9044-2d2be2ed6263'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:55.827592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:55.830272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86c88c79d5f4617981ee836b5d30d2b', 'request_id': 'req-9b7e0824-8a24-4904-9044-2d2be2ed6263'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:55.842815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86c88c79d5f4617981ee836b5d30d2b', 'request_id': 'req-9b7e0824-8a24-4904-9044-2d2be2ed6263'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:55.931463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] The token cache is being invalidate because project f86c88c79d5f4617981ee836b5d30d2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:55.932376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:55.933206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b7e0824-8a24-4904-9044-2d2be2ed6263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:55.938140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1092/2207] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:55 2022] DELETE /identity/v3/projects/f86c88c79d5f4617981ee836b5d30d2b => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:55.940338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232af662-cf4f-4a62-9173-0a36284e18e0 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:55.940820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232af662-cf4f-4a62-9173-0a36284e18e0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:55.940904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232af662-cf4f-4a62-9173-0a36284e18e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:55.963791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-232af662-cf4f-4a62-9173-0a36284e18e0 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:55.965021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-232af662-cf4f-4a62-9173-0a36284e18e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:56.034798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1116/2208] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:56.042773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359f1ba-eebc-4fd6-8f70-ac747ea422c3 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.044171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359f1ba-eebc-4fd6-8f70-ac747ea422c3 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.044171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359f1ba-eebc-4fd6-8f70-ac747ea422c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.062152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4359f1ba-eebc-4fd6-8f70-ac747ea422c3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:56.063361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4359f1ba-eebc-4fd6-8f70-ac747ea422c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:56.072952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1093/2209] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:56.083874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232af662-cf4f-4a62-9173-0a36284e18e0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.101669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232af662-cf4f-4a62-9173-0a36284e18e0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.106983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbb60e-1b00-4986-afc5-0bae3088c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.108418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbb60e-1b00-4986-afc5-0bae3088c1bf admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.108859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbb60e-1b00-4986-afc5-0bae3088c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.109137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbb60e-1b00-4986-afc5-0bae3088c1bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.110122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbb60e-1b00-4986-afc5-0bae3088c1bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:56.110657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbb60e-1b00-4986-afc5-0bae3088c1bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.112539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbb60e-1b00-4986-afc5-0bae3088c1bf admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.127712 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1117/2210] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:37:56.132425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359f1ba-eebc-4fd6-8f70-ac747ea422c3 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.142775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359f1ba-eebc-4fd6-8f70-ac747ea422c3 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.149758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2ed4a-e02e-4dcd-9d90-8327643bbd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.150425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2ed4a-e02e-4dcd-9d90-8327643bbd92 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.150705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2ed4a-e02e-4dcd-9d90-8327643bbd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.150978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2ed4a-e02e-4dcd-9d90-8327643bbd92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.151764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2ed4a-e02e-4dcd-9d90-8327643bbd92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.153806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2ed4a-e02e-4dcd-9d90-8327643bbd92 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.263738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1094/2211] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:56 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:56.267588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbb60e-1b00-4986-afc5-0bae3088c1bf admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.277106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2ed4a-e02e-4dcd-9d90-8327643bbd92 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.283977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9ea2ed4a-e02e-4dcd-9d90-8327643bbd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:56.287351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbb60e-1b00-4986-afc5-0bae3088c1bf admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.379437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2ed4a-e02e-4dcd-9d90-8327643bbd92 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.380620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280bfd5-8b2f-4bb8-81d5-7c837ffbb0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.381318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280bfd5-8b2f-4bb8-81d5-7c837ffbb0d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.381597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280bfd5-8b2f-4bb8-81d5-7c837ffbb0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.381873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280bfd5-8b2f-4bb8-81d5-7c837ffbb0d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.382568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280bfd5-8b2f-4bb8-81d5-7c837ffbb0d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.384438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.385007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280bfd5-8b2f-4bb8-81d5-7c837ffbb0d1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.398123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.398123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.398123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] PATH_INFO: `/v3/users/c08d0dba3e4442eda23f81261e2d40c2` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.398123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08d0dba3e4442eda23f81261e2d40c2)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.398123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.484794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1118/2212] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:56 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:56.486063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280bfd5-8b2f-4bb8-81d5-7c837ffbb0d1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.487594 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c280bfd5-8b2f-4bb8-81d5-7c837ffbb0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:56.502662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280bfd5-8b2f-4bb8-81d5-7c837ffbb0d1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.508347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.509350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.509657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.510003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] PATH_INFO: `/v3/users/3f47bf45459b4c4ea270900ce87ac38b` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.511494 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f47bf45459b4c4ea270900ce87ac38b)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.513531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.538514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:56.572973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08d0dba3e4442eda23f81261e2d40c2'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.573830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08d0dba3e4442eda23f81261e2d40c2'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.581139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08d0dba3e4442eda23f81261e2d40c2'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.581814 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08d0dba3e4442eda23f81261e2d40c2'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.595088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08d0dba3e4442eda23f81261e2d40c2'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.640398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f47bf45459b4c4ea270900ce87ac38b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.650051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1095/2213] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:56 2022] DELETE /identity/v3/users/c08d0dba3e4442eda23f81261e2d40c2 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:56.650951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.652441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:56.715611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab873ba-60b4-42eb-9768-4fe4b3a64377 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.718423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f47bf45459b4c4ea270900ce87ac38b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.719293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f47bf45459b4c4ea270900ce87ac38b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.720512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa946b-aece-4af0-a7d8-aacca9e07953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.721365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa946b-aece-4af0-a7d8-aacca9e07953 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.723755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa946b-aece-4af0-a7d8-aacca9e07953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.723755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa946b-aece-4af0-a7d8-aacca9e07953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.723755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa946b-aece-4af0-a7d8-aacca9e07953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.724259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa946b-aece-4af0-a7d8-aacca9e07953 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.727661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f47bf45459b4c4ea270900ce87ac38b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.728389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f47bf45459b4c4ea270900ce87ac38b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:56.735209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1096/2214] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:56.736469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4cb9-3074-4cf5-a4d9-c4aed919d853 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.736766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4cb9-3074-4cf5-a4d9-c4aed919d853 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.737042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4cb9-3074-4cf5-a4d9-c4aed919d853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.749336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1119/2215] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:56 2022] DELETE /identity/v3/users/3f47bf45459b4c4ea270900ce87ac38b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:56.750152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.751604 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:56.758589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-bd0c4cb9-3074-4cf5-a4d9-c4aed919d853 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:56.759764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-bd0c4cb9-3074-4cf5-a4d9-c4aed919d853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:56.815496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88be3c2-719d-4b56-b30c-ff70c4d5edca admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.817395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:56.819715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1097/2216] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:56.820627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9b470-c8b0-497f-9b5e-c2fa4dc96653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.820983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7990185-5e52-4cc5-a72c-97ba08bfb309 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.821242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7990185-5e52-4cc5-a72c-97ba08bfb309 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.821342 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9b470-c8b0-497f-9b5e-c2fa4dc96653 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.821520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7990185-5e52-4cc5-a72c-97ba08bfb309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.821632 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9b470-c8b0-497f-9b5e-c2fa4dc96653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.821883 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9b470-c8b0-497f-9b5e-c2fa4dc96653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.822617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9b470-c8b0-497f-9b5e-c2fa4dc96653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.824374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9b470-c8b0-497f-9b5e-c2fa4dc96653 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.838489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1120/2217] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:56.838489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9b470-c8b0-497f-9b5e-c2fa4dc96653 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.838489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-39c9b470-c8b0-497f-9b5e-c2fa4dc96653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:56.841006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-c7990185-5e52-4cc5-a72c-97ba08bfb309 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:56.842298 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c7990185-5e52-4cc5-a72c-97ba08bfb309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:56.850090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1098/2218] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:56.851393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7990185-5e52-4cc5-a72c-97ba08bfb309 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.861407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7990185-5e52-4cc5-a72c-97ba08bfb309 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.866226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18ad1-7bd2-4de4-bf30-e4bd57a5799c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.867112 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18ad1-7bd2-4de4-bf30-e4bd57a5799c admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.867444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18ad1-7bd2-4de4-bf30-e4bd57a5799c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.867758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18ad1-7bd2-4de4-bf30-e4bd57a5799c admin admin] PATH_INFO: `/v3/projects/463d49496646486883fc78b78f67232c/users/bc51b5e3810d4bb382942272926293dd/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.883395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18ad1-7bd2-4de4-bf30-e4bd57a5799c admin admin] RBAC: Authorizing `identity:create_grant(project_id=463d49496646486883fc78b78f67232c, user_id=bc51b5e3810d4bb382942272926293dd, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.886161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18ad1-7bd2-4de4-bf30-e4bd57a5799c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.897722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9b470-c8b0-497f-9b5e-c2fa4dc96653 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.901437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df16b0-29da-4472-888a-8106089f1165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.902083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df16b0-29da-4472-888a-8106089f1165 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.902362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df16b0-29da-4472-888a-8106089f1165 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.904808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df16b0-29da-4472-888a-8106089f1165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.911740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df16b0-29da-4472-888a-8106089f1165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.913196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df16b0-29da-4472-888a-8106089f1165 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.922799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1121/2219] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:56.923585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df16b0-29da-4472-888a-8106089f1165 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:56.931153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df16b0-29da-4472-888a-8106089f1165 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:56.934975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5addaf-f73d-43a6-b3b8-d72e5580839c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:56.935660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5addaf-f73d-43a6-b3b8-d72e5580839c service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.935945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5addaf-f73d-43a6-b3b8-d72e5580839c service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.936218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5addaf-f73d-43a6-b3b8-d72e5580839c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.946226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5addaf-f73d-43a6-b3b8-d72e5580839c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:56.947658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5addaf-f73d-43a6-b3b8-d72e5580839c service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:56.957466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1122/2220] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:56.960427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1099/2221] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:56 2022] PUT /identity/v3/projects/463d49496646486883fc78b78f67232c/users/bc51b5e3810d4bb382942272926293dd/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:56.971805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7a2f-1fff-495b-8838-0a1f86559fbd None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:56.972095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7a2f-1fff-495b-8838-0a1f86559fbd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:56.972370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7a2f-1fff-495b-8838-0a1f86559fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:56.998443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-724e7a2f-1fff-495b-8838-0a1f86559fbd None None] MFA Rules not processed for user `bc51b5e3810d4bb382942272926293dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:56.999693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-724e7a2f-1fff-495b-8838-0a1f86559fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:57.016928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:57.070050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1123/2222] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:57.081722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18ad1-7bd2-4de4-bf30-e4bd57a5799c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:57.083640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-bda18ad1-7bd2-4de4-bf30-e4bd57a5799c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:57.140117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18ad1-7bd2-4de4-bf30-e4bd57a5799c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:57.143758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe19d7-4ba6-4a4a-a0d7-918ea9753284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:57.144405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe19d7-4ba6-4a4a-a0d7-918ea9753284 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:57.144686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe19d7-4ba6-4a4a-a0d7-918ea9753284 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:57.144958 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe19d7-4ba6-4a4a-a0d7-918ea9753284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:57.154601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe19d7-4ba6-4a4a-a0d7-918ea9753284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:57.156121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe19d7-4ba6-4a4a-a0d7-918ea9753284 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:57.166027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1100/2223] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:37:57.464203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe19d7-4ba6-4a4a-a0d7-918ea9753284 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:57.464654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e7a2f-1fff-495b-8838-0a1f86559fbd None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:57.465723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1afe19d7-4ba6-4a4a-a0d7-918ea9753284 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:57.466331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-724e7a2f-1fff-495b-8838-0a1f86559fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:57.537415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:57.567447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e7a2f-1fff-495b-8838-0a1f86559fbd None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:57.577441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:57.577656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe19d7-4ba6-4a4a-a0d7-918ea9753284 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:57.578381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:57.578668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:57.578943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] PATH_INFO: `/v3/projects/e301883825434ae0b68261b3dae4a323` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:57.580885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e301883825434ae0b68261b3dae4a323)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:57.583496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:57.584355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:57.585518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:57.585897 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:57.586233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] PATH_INFO: `/v3/projects/938847a2e4de4e5e9721172d81121879` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:57.587966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=938847a2e4de4e5e9721172d81121879)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:57.594622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:57.703721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '938847a2e4de4e5e9721172d81121879', 'request_id': 'req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.710094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '938847a2e4de4e5e9721172d81121879', 'request_id': 'req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.711731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e301883825434ae0b68261b3dae4a323', 'request_id': 'req-3703fc29-aa78-4897-ae81-cc9dc3d76f31'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.717539 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '938847a2e4de4e5e9721172d81121879', 'request_id': 'req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.769585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e301883825434ae0b68261b3dae4a323', 'request_id': 'req-3703fc29-aa78-4897-ae81-cc9dc3d76f31'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.775979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e301883825434ae0b68261b3dae4a323', 'request_id': 'req-3703fc29-aa78-4897-ae81-cc9dc3d76f31'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.794601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] The token cache is being invalidate because project e301883825434ae0b68261b3dae4a323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:57.794601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.794601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3703fc29-aa78-4897-ae81-cc9dc3d76f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.794601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1124/2224] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:57 2022] DELETE /identity/v3/projects/e301883825434ae0b68261b3dae4a323 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:57.797474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e580f86-c17c-40ff-99c7-7ea189a8ef49 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:57.797763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e580f86-c17c-40ff-99c7-7ea189a8ef49 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:57.798041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e580f86-c17c-40ff-99c7-7ea189a8ef49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:57.816937 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-6e580f86-c17c-40ff-99c7-7ea189a8ef49 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:57.818140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6e580f86-c17c-40ff-99c7-7ea189a8ef49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:57.831724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] The token cache is being invalidate because project 938847a2e4de4e5e9721172d81121879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:57.832428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.832634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:57.834206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1101/2225] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:57 2022] DELETE /identity/v3/projects/938847a2e4de4e5e9721172d81121879 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:57.838065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:57.839710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:57.860360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:57.878556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1125/2226] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:57.886999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773846c-2628-4680-bd7e-96fe783d6b64 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:57.887279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773846c-2628-4680-bd7e-96fe783d6b64 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:57.887656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773846c-2628-4680-bd7e-96fe783d6b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:57.905509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3773846c-2628-4680-bd7e-96fe783d6b64 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:57.906598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7b71a-b6f4-42e5-bf82-7ccca1562dc3 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:57.906773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3773846c-2628-4680-bd7e-96fe783d6b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:57.911372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:57.913050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:57.913050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:57.913405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] PATH_INFO: `/v3/users/9b3fdd32caed47d98027c658408bf2c5` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:57.914978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b3fdd32caed47d98027c658408bf2c5)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:57.915584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1126/2227] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:57.916962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:57.919887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773846c-2628-4680-bd7e-96fe783d6b64 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:57.929388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773846c-2628-4680-bd7e-96fe783d6b64 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:57.934198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6a86a-e083-4788-a697-82d498456b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:57.934848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6a86a-e083-4788-a697-82d498456b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:57.935131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6a86a-e083-4788-a697-82d498456b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:57.944154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6a86a-e083-4788-a697-82d498456b68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:57.946071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6a86a-e083-4788-a697-82d498456b68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:57.946492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6a86a-e083-4788-a697-82d498456b68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:57.948497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6a86a-e083-4788-a697-82d498456b68 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:57.962360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1127/2228] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:37:57.966398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6a86a-e083-4788-a697-82d498456b68 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:57.977074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6a86a-e083-4788-a697-82d498456b68 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:57.981940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b219f-05bd-40b0-9235-5f6d821d8a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:57.982591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b219f-05bd-40b0-9235-5f6d821d8a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:57.982872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b219f-05bd-40b0-9235-5f6d821d8a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:57.983145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b219f-05bd-40b0-9235-5f6d821d8a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:57.983878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b219f-05bd-40b0-9235-5f6d821d8a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:57.985795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b219f-05bd-40b0-9235-5f6d821d8a46 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:58.056735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1128/2229] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:57 2022] POST /identity/v3/projects => generated 391 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:37:58.060900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b219f-05bd-40b0-9235-5f6d821d8a46 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:58.120842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b3fdd32caed47d98027c658408bf2c5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:58.121520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b3fdd32caed47d98027c658408bf2c5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:58.123471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b219f-05bd-40b0-9235-5f6d821d8a46 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:58.128411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f175aa2-b916-444f-8851-c0cc337712fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:58.128626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b3fdd32caed47d98027c658408bf2c5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:58.129080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f175aa2-b916-444f-8851-c0cc337712fd admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:58.129271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b3fdd32caed47d98027c658408bf2c5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:58.129394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f175aa2-b916-444f-8851-c0cc337712fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:58.129618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f175aa2-b916-444f-8851-c0cc337712fd admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:58.130308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f175aa2-b916-444f-8851-c0cc337712fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:58.132440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f175aa2-b916-444f-8851-c0cc337712fd admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:58.141542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b3fdd32caed47d98027c658408bf2c5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:58.143213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:58.267303 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1102/2230] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:37:57 2022] DELETE /identity/v3/users/9b3fdd32caed47d98027c658408bf2c5 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:58.269333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1129/2231] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:58 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:58.273933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:58.277459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:58.355648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d98916-dbcd-4412-b9d1-9f0775d89996 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:58.360932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10581e76-8e88-4a40-92bb-1a83194fea23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:58.361611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10581e76-8e88-4a40-92bb-1a83194fea23 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:58.361899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10581e76-8e88-4a40-92bb-1a83194fea23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:58.362175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10581e76-8e88-4a40-92bb-1a83194fea23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:58.362914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10581e76-8e88-4a40-92bb-1a83194fea23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:58.364781 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10581e76-8e88-4a40-92bb-1a83194fea23 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:58.377229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1103/2232] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:58.381053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f175aa2-b916-444f-8851-c0cc337712fd admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:58.391966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f175aa2-b916-444f-8851-c0cc337712fd admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:58.396832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60f4a1-6781-4b5b-ae61-72aa8cea4b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:58.398081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f4a1-6781-4b5b-ae61-72aa8cea4b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:58.399078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f4a1-6781-4b5b-ae61-72aa8cea4b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:58.399078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60f4a1-6781-4b5b-ae61-72aa8cea4b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:58.399565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60f4a1-6781-4b5b-ae61-72aa8cea4b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:58.401172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60f4a1-6781-4b5b-ae61-72aa8cea4b88 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:58.411274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1130/2233] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:58.415840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10581e76-8e88-4a40-92bb-1a83194fea23 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:58.425900 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10581e76-8e88-4a40-92bb-1a83194fea23 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:58.430658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c49f5-988c-4eff-927d-5ab6527c5a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:58.431560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c49f5-988c-4eff-927d-5ab6527c5a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:58.431851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c49f5-988c-4eff-927d-5ab6527c5a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:58.432129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c49f5-988c-4eff-927d-5ab6527c5a4d admin admin] PATH_INFO: `/v3/projects/b9185bb22ed4471aaca5af5422e378c0/users/2f53544224d343a88ca7d2371f4fb1bb/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:58.448821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c49f5-988c-4eff-927d-5ab6527c5a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9185bb22ed4471aaca5af5422e378c0, user_id=2f53544224d343a88ca7d2371f4fb1bb, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:58.451565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c49f5-988c-4eff-927d-5ab6527c5a4d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:58.520368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1104/2234] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:58 2022] PUT /identity/v3/projects/b9185bb22ed4471aaca5af5422e378c0/users/2f53544224d343a88ca7d2371f4fb1bb/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:58.524838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ee78-3613-4e05-9247-8d16e8b4bbb1 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:58.525719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ee78-3613-4e05-9247-8d16e8b4bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:58.525719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ee78-3613-4e05-9247-8d16e8b4bbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:58.556947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ecd3ee78-3613-4e05-9247-8d16e8b4bbb1 None None] MFA Rules not processed for user `2f53544224d343a88ca7d2371f4fb1bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:58.558196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ecd3ee78-3613-4e05-9247-8d16e8b4bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:58.682582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1131/2235] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:58.690859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c49f5-988c-4eff-927d-5ab6527c5a4d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:58.705438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-084c49f5-988c-4eff-927d-5ab6527c5a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:58.752105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3ee78-3613-4e05-9247-8d16e8b4bbb1 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:58.754043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ecd3ee78-3613-4e05-9247-8d16e8b4bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:58.755892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c49f5-988c-4eff-927d-5ab6527c5a4d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:58.760888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49326b85-be74-4d05-9d24-21c87aa08478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:58.761708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326b85-be74-4d05-9d24-21c87aa08478 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:58.762085 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326b85-be74-4d05-9d24-21c87aa08478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:58.762468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326b85-be74-4d05-9d24-21c87aa08478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:58.774730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49326b85-be74-4d05-9d24-21c87aa08478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:58.776924 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49326b85-be74-4d05-9d24-21c87aa08478 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:58.781955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:58.789870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1105/2236] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:37:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:37:58.851814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3ee78-3613-4e05-9247-8d16e8b4bbb1 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:58.860800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:58.861954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:58.862156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:58.862457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] PATH_INFO: `/v3/projects/c53560294e954b1888c489d4ae243fdb` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:58.863732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c53560294e954b1888c489d4ae243fdb)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:58.866223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:58.975193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c53560294e954b1888c489d4ae243fdb', 'request_id': 'req-a03b2c82-c4e9-4418-9bb9-535232e8f54c'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:59.056717 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c53560294e954b1888c489d4ae243fdb', 'request_id': 'req-a03b2c82-c4e9-4418-9bb9-535232e8f54c'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:59.068547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c53560294e954b1888c489d4ae243fdb', 'request_id': 'req-a03b2c82-c4e9-4418-9bb9-535232e8f54c'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:59.087183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] The token cache is being invalidate because project c53560294e954b1888c489d4ae243fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:37:59.087742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:59.088615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a03b2c82-c4e9-4418-9bb9-535232e8f54c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:37:59.090498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1132/2237] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:37:58 2022] DELETE /identity/v3/projects/c53560294e954b1888c489d4ae243fdb => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:37:59.096819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917a689-d3e6-4df8-bdef-867eb3ab8161 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.097393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917a689-d3e6-4df8-bdef-867eb3ab8161 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.097807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917a689-d3e6-4df8-bdef-867eb3ab8161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.127779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b917a689-d3e6-4df8-bdef-867eb3ab8161 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:59.129077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b917a689-d3e6-4df8-bdef-867eb3ab8161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:59.204182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1106/2238] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:37:59.211384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d41ee-6526-4dc0-91ca-2167fb54bf85 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.211715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d41ee-6526-4dc0-91ca-2167fb54bf85 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.211995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d41ee-6526-4dc0-91ca-2167fb54bf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.232033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-065d41ee-6526-4dc0-91ca-2167fb54bf85 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:59.233256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-065d41ee-6526-4dc0-91ca-2167fb54bf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:59.247257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1133/2239] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:59.248538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917a689-d3e6-4df8-bdef-867eb3ab8161 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:59.258367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917a689-d3e6-4df8-bdef-867eb3ab8161 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:59.263505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc143e-0e89-4a0d-9b67-c9797485b162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:59.264308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc143e-0e89-4a0d-9b67-c9797485b162 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.264592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc143e-0e89-4a0d-9b67-c9797485b162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.264883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc143e-0e89-4a0d-9b67-c9797485b162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.265637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc143e-0e89-4a0d-9b67-c9797485b162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:37:59.265939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc143e-0e89-4a0d-9b67-c9797485b162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:59.267542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc143e-0e89-4a0d-9b67-c9797485b162 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:59.271511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:59.280748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1107/2240] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:37:59.284874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d41ee-6526-4dc0-91ca-2167fb54bf85 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:59.297045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d41ee-6526-4dc0-91ca-2167fb54bf85 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:59.302410 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594d20e-e537-4e0a-8ae3-bace075f4264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:59.302768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594d20e-e537-4e0a-8ae3-bace075f4264 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.303682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594d20e-e537-4e0a-8ae3-bace075f4264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.303973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594d20e-e537-4e0a-8ae3-bace075f4264 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.304687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594d20e-e537-4e0a-8ae3-bace075f4264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:59.307052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594d20e-e537-4e0a-8ae3-bace075f4264 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:59.407990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1134/2241] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:59 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:59.415331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc143e-0e89-4a0d-9b67-c9797485b162 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:59.425850 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc143e-0e89-4a0d-9b67-c9797485b162 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:59.482980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e22f4c-1801-4383-b4ea-44f5aef0a078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:59.483906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e22f4c-1801-4383-b4ea-44f5aef0a078 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.484140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e22f4c-1801-4383-b4ea-44f5aef0a078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.484420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e22f4c-1801-4383-b4ea-44f5aef0a078 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.485112 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e22f4c-1801-4383-b4ea-44f5aef0a078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:59.487044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e22f4c-1801-4383-b4ea-44f5aef0a078 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:59.548401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1108/2242] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:37:59 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:37:59.551187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594d20e-e537-4e0a-8ae3-bace075f4264 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:59.561793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594d20e-e537-4e0a-8ae3-bace075f4264 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:59.566519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5e483-88b0-41ac-a484-dd9e012c34f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:59.567185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5e483-88b0-41ac-a484-dd9e012c34f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.567487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5e483-88b0-41ac-a484-dd9e012c34f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.567785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5e483-88b0-41ac-a484-dd9e012c34f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.568560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5e483-88b0-41ac-a484-dd9e012c34f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:59.570195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5e483-88b0-41ac-a484-dd9e012c34f1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:59.582224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1135/2243] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:37:59.587004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e22f4c-1801-4383-b4ea-44f5aef0a078 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:59.598788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e22f4c-1801-4383-b4ea-44f5aef0a078 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:59.603041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e81a20-290a-4ca5-bfac-d169d064b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:59.603828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e81a20-290a-4ca5-bfac-d169d064b228 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.604111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e81a20-290a-4ca5-bfac-d169d064b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.604476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e81a20-290a-4ca5-bfac-d169d064b228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.605106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e81a20-290a-4ca5-bfac-d169d064b228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:59.606840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e81a20-290a-4ca5-bfac-d169d064b228 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:59.619521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1109/2244] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:37:59.623208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5e483-88b0-41ac-a484-dd9e012c34f1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:59.633928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5e483-88b0-41ac-a484-dd9e012c34f1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:59.638925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd26da-c067-4eba-8fb1-d6e46c4d18c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:59.639958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd26da-c067-4eba-8fb1-d6e46c4d18c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.640303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd26da-c067-4eba-8fb1-d6e46c4d18c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.640639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd26da-c067-4eba-8fb1-d6e46c4d18c3 admin admin] PATH_INFO: `/v3/projects/9c0f9e40a8d240ad9074e0e876b21fe0/users/b31a253f60094a0b9f2e31cb4f4d2948/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.658436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd26da-c067-4eba-8fb1-d6e46c4d18c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0f9e40a8d240ad9074e0e876b21fe0, user_id=b31a253f60094a0b9f2e31cb4f4d2948, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:59.661386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd26da-c067-4eba-8fb1-d6e46c4d18c3 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:37:59.752952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1136/2245] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:37:59 2022] PUT /identity/v3/projects/9c0f9e40a8d240ad9074e0e876b21fe0/users/b31a253f60094a0b9f2e31cb4f4d2948/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:37:59.764839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f39d9-5741-4c43-a995-aa0ee23612c2 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.764839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f39d9-5741-4c43-a995-aa0ee23612c2 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.764839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f39d9-5741-4c43-a995-aa0ee23612c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.791630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-565f39d9-5741-4c43-a995-aa0ee23612c2 None None] MFA Rules not processed for user `b31a253f60094a0b9f2e31cb4f4d2948`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:37:59.792862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-565f39d9-5741-4c43-a995-aa0ee23612c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:37:59.866393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1110/2246] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:37:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:37:59.870259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd26da-c067-4eba-8fb1-d6e46c4d18c3 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:37:59.918302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:37:59.937195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd26da-c067-4eba-8fb1-d6e46c4d18c3 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:37:59.942011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3fe8a-7bff-416d-8417-836578f2fe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:37:59.942678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3fe8a-7bff-416d-8417-836578f2fe77 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:37:59.942955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3fe8a-7bff-416d-8417-836578f2fe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:37:59.943227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3fe8a-7bff-416d-8417-836578f2fe77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:37:59.944476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3fe8a-7bff-416d-8417-836578f2fe77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:37:59.946506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3fe8a-7bff-416d-8417-836578f2fe77 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:00.075065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1137/2247] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:37:59 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:38:00.078987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f39d9-5741-4c43-a995-aa0ee23612c2 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:00.105161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f39d9-5741-4c43-a995-aa0ee23612c2 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:00.179976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3de7bc-462a-45ae-8c52-e525f6fd5ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:00.181059 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3de7bc-462a-45ae-8c52-e525f6fd5ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:00.181144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3de7bc-462a-45ae-8c52-e525f6fd5ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:00.183711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3de7bc-462a-45ae-8c52-e525f6fd5ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:00.183711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3de7bc-462a-45ae-8c52-e525f6fd5ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:00.184604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3de7bc-462a-45ae-8c52-e525f6fd5ce3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:00.280860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1111/2248] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:38:00 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:00.282402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3fe8a-7bff-416d-8417-836578f2fe77 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:00.304064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3fe8a-7bff-416d-8417-836578f2fe77 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:00.312685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4058375-fe12-421a-b5e3-ab598b2282cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:00.314942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4058375-fe12-421a-b5e3-ab598b2282cc admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:00.314942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4058375-fe12-421a-b5e3-ab598b2282cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:00.314942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4058375-fe12-421a-b5e3-ab598b2282cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:00.314942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4058375-fe12-421a-b5e3-ab598b2282cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:00.317155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4058375-fe12-421a-b5e3-ab598b2282cc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:00.333943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1138/2249] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:38:00.338220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3de7bc-462a-45ae-8c52-e525f6fd5ce3 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:00.348915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3de7bc-462a-45ae-8c52-e525f6fd5ce3 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:00.353732 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd33d9-c800-4514-b82d-17bfbd061663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:00.354604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd33d9-c800-4514-b82d-17bfbd061663 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:00.354883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd33d9-c800-4514-b82d-17bfbd061663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:00.355288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd33d9-c800-4514-b82d-17bfbd061663 admin admin] PATH_INFO: `/v3/projects/75ff8fc577fa4bd190ddd254cb03fb95/users/9d533588b29c4c03a81097ed6e9be9d1/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:00.370608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd33d9-c800-4514-b82d-17bfbd061663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ff8fc577fa4bd190ddd254cb03fb95, user_id=9d533588b29c4c03a81097ed6e9be9d1, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:00.373471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd33d9-c800-4514-b82d-17bfbd061663 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:00.485008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1112/2250] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:00 2022] PUT /identity/v3/projects/75ff8fc577fa4bd190ddd254cb03fb95/users/9d533588b29c4c03a81097ed6e9be9d1/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:00.489122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4058375-fe12-421a-b5e3-ab598b2282cc admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:00.499889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4058375-fe12-421a-b5e3-ab598b2282cc admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:00.555688 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c51b6-f03d-40ea-9bb1-c8051196c4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:00.556350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c51b6-f03d-40ea-9bb1-c8051196c4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:00.556630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c51b6-f03d-40ea-9bb1-c8051196c4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:00.556903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c51b6-f03d-40ea-9bb1-c8051196c4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:00.557614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c51b6-f03d-40ea-9bb1-c8051196c4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:00.559269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c51b6-f03d-40ea-9bb1-c8051196c4ce admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:00.570362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1139/2251] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:00.573509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd33d9-c800-4514-b82d-17bfbd061663 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:00.583776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd33d9-c800-4514-b82d-17bfbd061663 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:00.588448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3f954-9e04-49da-9a07-5c928423abc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:00.589346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3f954-9e04-49da-9a07-5c928423abc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:00.589613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3f954-9e04-49da-9a07-5c928423abc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:00.589887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3f954-9e04-49da-9a07-5c928423abc1 admin admin] PATH_INFO: `/v3/projects/75ff8fc577fa4bd190ddd254cb03fb95/users/9d533588b29c4c03a81097ed6e9be9d1/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:00.591927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3f954-9e04-49da-9a07-5c928423abc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ff8fc577fa4bd190ddd254cb03fb95, user_id=9d533588b29c4c03a81097ed6e9be9d1, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:00.594564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3f954-9e04-49da-9a07-5c928423abc1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:00.605068 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1113/2252] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:00 2022] PUT /identity/v3/projects/75ff8fc577fa4bd190ddd254cb03fb95/users/9d533588b29c4c03a81097ed6e9be9d1/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:00.616403 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592101d-b556-4c50-b4ba-a87a00d290a4 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:00.616691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592101d-b556-4c50-b4ba-a87a00d290a4 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:00.622530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592101d-b556-4c50-b4ba-a87a00d290a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:00.648342 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-b592101d-b556-4c50-b4ba-a87a00d290a4 None None] MFA Rules not processed for user `9d533588b29c4c03a81097ed6e9be9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:00.649623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b592101d-b556-4c50-b4ba-a87a00d290a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:00.729190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1140/2253] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:00.746342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3f954-9e04-49da-9a07-5c928423abc1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:00.746342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-54f3f954-9e04-49da-9a07-5c928423abc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:00.801523 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3f954-9e04-49da-9a07-5c928423abc1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:00.805751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181172d3-eefe-4fc2-9673-f7027a675dd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:00.806395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181172d3-eefe-4fc2-9673-f7027a675dd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:00.806674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181172d3-eefe-4fc2-9673-f7027a675dd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:00.806945 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181172d3-eefe-4fc2-9673-f7027a675dd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:00.816167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181172d3-eefe-4fc2-9673-f7027a675dd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:00.817588 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181172d3-eefe-4fc2-9673-f7027a675dd1 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:00.832991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1114/2254] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:00.927671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592101d-b556-4c50-b4ba-a87a00d290a4 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:00.929128 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b592101d-b556-4c50-b4ba-a87a00d290a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:00.941662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592101d-b556-4c50-b4ba-a87a00d290a4 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:00.945992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d1e42-e714-41b4-9b8a-09920440cae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:00.946622 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d1e42-e714-41b4-9b8a-09920440cae1 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:00.946901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d1e42-e714-41b4-9b8a-09920440cae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:00.947172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d1e42-e714-41b4-9b8a-09920440cae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:01.001658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d1e42-e714-41b4-9b8a-09920440cae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:01.003115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d1e42-e714-41b4-9b8a-09920440cae1 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:01.012532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1141/2255] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:01.078369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:01.998336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:12.152245 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:12.649174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=12 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:14.671758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181172d3-eefe-4fc2-9673-f7027a675dd1 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:14.744105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181172d3-eefe-4fc2-9673-f7027a675dd1 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:14.748875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:14.749879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:14.750196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:14.750445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] PATH_INFO: `/v3/users/b31a253f60094a0b9f2e31cb4f4d2948` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:14.751701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31a253f60094a0b9f2e31cb4f4d2948)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:14.753621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:15.005642 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31a253f60094a0b9f2e31cb4f4d2948'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:15.006319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31a253f60094a0b9f2e31cb4f4d2948'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:15.013939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31a253f60094a0b9f2e31cb4f4d2948'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:15.014572 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31a253f60094a0b9f2e31cb4f4d2948'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:15.027074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31a253f60094a0b9f2e31cb4f4d2948'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:15.089810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1115/2256] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:38:14 2022] DELETE /identity/v3/users/b31a253f60094a0b9f2e31cb4f4d2948 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:15.094651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b2360-a761-4f6f-966b-3306870fa0b5 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:15.095182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b2360-a761-4f6f-966b-3306870fa0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:15.095283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b2360-a761-4f6f-966b-3306870fa0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:15.115817 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-064b2360-a761-4f6f-966b-3306870fa0b5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:15.117121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-064b2360-a761-4f6f-966b-3306870fa0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:15.181947 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1142/2257] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:15.191434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:15.209246 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:15.257462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a440213-22ba-4965-9dc9-6a95be9290ce admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:15.262110 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486602e3-bab3-4df1-bef5-be8cd443ee5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:15.262760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486602e3-bab3-4df1-bef5-be8cd443ee5f service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:15.263043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486602e3-bab3-4df1-bef5-be8cd443ee5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:15.263318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486602e3-bab3-4df1-bef5-be8cd443ee5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:15.276041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486602e3-bab3-4df1-bef5-be8cd443ee5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:15.277552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486602e3-bab3-4df1-bef5-be8cd443ee5f service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:15.288494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1116/2258] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:38:15.829157 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b2360-a761-4f6f-966b-3306870fa0b5 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:15.831178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-064b2360-a761-4f6f-966b-3306870fa0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:15.845468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b2360-a761-4f6f-966b-3306870fa0b5 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:15.851959 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:15.853155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:15.853493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:15.854473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] PATH_INFO: `/v3/projects/9c0f9e40a8d240ad9074e0e876b21fe0` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:15.855988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0f9e40a8d240ad9074e0e876b21fe0)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:15.858130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:15.989034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0f9e40a8d240ad9074e0e876b21fe0', 'request_id': 'req-ddda4c2f-aaf4-451f-a377-3f98661726fc'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.083191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0f9e40a8d240ad9074e0e876b21fe0', 'request_id': 'req-ddda4c2f-aaf4-451f-a377-3f98661726fc'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.098724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0f9e40a8d240ad9074e0e876b21fe0', 'request_id': 'req-ddda4c2f-aaf4-451f-a377-3f98661726fc'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.113037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] The token cache is being invalidate because project 9c0f9e40a8d240ad9074e0e876b21fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:38:16.115840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.115840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.120866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1143/2259] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:38:15 2022] DELETE /identity/v3/projects/9c0f9e40a8d240ad9074e0e876b21fe0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:16.121518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486602e3-bab3-4df1-bef5-be8cd443ee5f service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:16.133505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486602e3-bab3-4df1-bef5-be8cd443ee5f service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:16.170515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:16.209731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:16.210613 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:16.210896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:16.211201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] PATH_INFO: `/v3/users/9d533588b29c4c03a81097ed6e9be9d1` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:16.212520 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d533588b29c4c03a81097ed6e9be9d1)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:16.214569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:16.406160 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d533588b29c4c03a81097ed6e9be9d1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.406826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d533588b29c4c03a81097ed6e9be9d1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.413771 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d533588b29c4c03a81097ed6e9be9d1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.414421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d533588b29c4c03a81097ed6e9be9d1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.427261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-66902704-ded3-45d4-b471-f49a6881139c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d533588b29c4c03a81097ed6e9be9d1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:16.501709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1117/2260] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:38:16 2022] DELETE /identity/v3/users/9d533588b29c4c03a81097ed6e9be9d1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:17.002961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:17.002961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:17.102206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda4c2f-aaf4-451f-a377-3f98661726fc admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:17.115576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:17.156086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:17.161297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:17.161297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:17.161297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] PATH_INFO: `/v3/projects/75ff8fc577fa4bd190ddd254cb03fb95` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:17.161297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ff8fc577fa4bd190ddd254cb03fb95)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:17.161297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:17.279047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ff8fc577fa4bd190ddd254cb03fb95', 'request_id': 'req-a503281e-ca49-4715-ae1e-1a3058008ee8'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:17.352354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ff8fc577fa4bd190ddd254cb03fb95', 'request_id': 'req-a503281e-ca49-4715-ae1e-1a3058008ee8'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:17.358950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ff8fc577fa4bd190ddd254cb03fb95', 'request_id': 'req-a503281e-ca49-4715-ae1e-1a3058008ee8'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:17.372629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] The token cache is being invalidate because project 75ff8fc577fa4bd190ddd254cb03fb95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:38:17.375705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:17.375705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a503281e-ca49-4715-ae1e-1a3058008ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:17.375705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1144/2261] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:38:16 2022] DELETE /identity/v3/projects/75ff8fc577fa4bd190ddd254cb03fb95 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:17.380533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73ca67-a14b-4184-9321-ca431449626a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:17.381663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73ca67-a14b-4184-9321-ca431449626a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:17.382043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73ca67-a14b-4184-9321-ca431449626a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:17.441187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-0c73ca67-a14b-4184-9321-ca431449626a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:17.442500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0c73ca67-a14b-4184-9321-ca431449626a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:17.512282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1118/2262] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:17.520011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652297c5-e8d7-4980-ae59-442b5cb51d18 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:17.520385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652297c5-e8d7-4980-ae59-442b5cb51d18 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:17.520662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652297c5-e8d7-4980-ae59-442b5cb51d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:17.541710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-652297c5-e8d7-4980-ae59-442b5cb51d18 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:17.543037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-652297c5-e8d7-4980-ae59-442b5cb51d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:17.556628 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1145/2263] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:17.560982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73ca67-a14b-4184-9321-ca431449626a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:17.570650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73ca67-a14b-4184-9321-ca431449626a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:17.575255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d813abf-164a-4085-9f6b-b258b9b271fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:17.575942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d813abf-164a-4085-9f6b-b258b9b271fc admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:17.576253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d813abf-164a-4085-9f6b-b258b9b271fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:17.576529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d813abf-164a-4085-9f6b-b258b9b271fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:17.577256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d813abf-164a-4085-9f6b-b258b9b271fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:38:17.577585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d813abf-164a-4085-9f6b-b258b9b271fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:17.579182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d813abf-164a-4085-9f6b-b258b9b271fc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:17.594232 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1119/2264] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:38:17.595578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652297c5-e8d7-4980-ae59-442b5cb51d18 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:17.605633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652297c5-e8d7-4980-ae59-442b5cb51d18 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:17.610774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e2cee-6d69-4e76-8260-5c00d883f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:17.611428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e2cee-6d69-4e76-8260-5c00d883f988 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:17.611735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e2cee-6d69-4e76-8260-5c00d883f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:17.612009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e2cee-6d69-4e76-8260-5c00d883f988 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:17.612707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e2cee-6d69-4e76-8260-5c00d883f988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:17.614607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e2cee-6d69-4e76-8260-5c00d883f988 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:17.701719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1146/2265] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:38:17 2022] POST /identity/v3/projects => generated 407 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:17.706161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d813abf-164a-4085-9f6b-b258b9b271fc admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:17.716344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d813abf-164a-4085-9f6b-b258b9b271fc admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:17.720049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:17.772700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfc770-8ce8-458d-9ede-f3213c09b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:17.773407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfc770-8ce8-458d-9ede-f3213c09b11f admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:17.773686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfc770-8ce8-458d-9ede-f3213c09b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:17.773960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfc770-8ce8-458d-9ede-f3213c09b11f admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:17.774799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfc770-8ce8-458d-9ede-f3213c09b11f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:17.776757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfc770-8ce8-458d-9ede-f3213c09b11f admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:17.864000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1120/2266] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:38:17 2022] POST /identity/v3/users => generated 304 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:38:17.868596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e2cee-6d69-4e76-8260-5c00d883f988 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:17.879064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e2cee-6d69-4e76-8260-5c00d883f988 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:17.883893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ea736-8aef-4fc7-ac46-29f98a4d4970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:17.884565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ea736-8aef-4fc7-ac46-29f98a4d4970 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:17.884845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ea736-8aef-4fc7-ac46-29f98a4d4970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:17.885122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ea736-8aef-4fc7-ac46-29f98a4d4970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:17.885874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ea736-8aef-4fc7-ac46-29f98a4d4970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:17.887481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ea736-8aef-4fc7-ac46-29f98a4d4970 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:17.898469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1147/2267] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:38:17.912311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfc770-8ce8-458d-9ede-f3213c09b11f admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:17.923499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfc770-8ce8-458d-9ede-f3213c09b11f admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:17.928743 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3bb73-e379-4381-89a1-ef38178d904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:17.929419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3bb73-e379-4381-89a1-ef38178d904e admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:17.929712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3bb73-e379-4381-89a1-ef38178d904e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:17.930061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3bb73-e379-4381-89a1-ef38178d904e admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:17.930803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3bb73-e379-4381-89a1-ef38178d904e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:17.932466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3bb73-e379-4381-89a1-ef38178d904e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:17.946475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1121/2268] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:17.950725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ea736-8aef-4fc7-ac46-29f98a4d4970 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:17.962006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ea736-8aef-4fc7-ac46-29f98a4d4970 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:17.967378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069c954-bc5a-4196-b0ff-e901ed54cbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:17.968535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069c954-bc5a-4196-b0ff-e901ed54cbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:17.968907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069c954-bc5a-4196-b0ff-e901ed54cbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:17.969250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069c954-bc5a-4196-b0ff-e901ed54cbf5 admin admin] PATH_INFO: `/v3/projects/6ca0a71cf4c640ce88e4d686aa1c91b1/users/3568fe48ba0c46739d8bccd988bacaba/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:17.985926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069c954-bc5a-4196-b0ff-e901ed54cbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca0a71cf4c640ce88e4d686aa1c91b1, user_id=3568fe48ba0c46739d8bccd988bacaba, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:17.988766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069c954-bc5a-4196-b0ff-e901ed54cbf5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:18.081097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1148/2269] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:17 2022] PUT /identity/v3/projects/6ca0a71cf4c640ce88e4d686aa1c91b1/users/3568fe48ba0c46739d8bccd988bacaba/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:18.084285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8c3aa-6a53-4f4b-a063-e580c3665bb7 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:18.084581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8c3aa-6a53-4f4b-a063-e580c3665bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:18.084861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8c3aa-6a53-4f4b-a063-e580c3665bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:18.113609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-80b8c3aa-6a53-4f4b-a063-e580c3665bb7 None None] MFA Rules not processed for user `3568fe48ba0c46739d8bccd988bacaba`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:18.114849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-80b8c3aa-6a53-4f4b-a063-e580c3665bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:18.182938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1122/2270] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:18 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:18.187323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069c954-bc5a-4196-b0ff-e901ed54cbf5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:18.244583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:18.257425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069c954-bc5a-4196-b0ff-e901ed54cbf5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:18.262093 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d602f32-69d8-4ac1-8b87-33428780ece9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:18.262744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d602f32-69d8-4ac1-8b87-33428780ece9 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:18.263026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d602f32-69d8-4ac1-8b87-33428780ece9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:18.263300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d602f32-69d8-4ac1-8b87-33428780ece9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:18.264029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d602f32-69d8-4ac1-8b87-33428780ece9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:18.265922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d602f32-69d8-4ac1-8b87-33428780ece9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:18.377014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1149/2271] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:38:18 2022] POST /identity/v3/projects => generated 407 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:18.381011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8c3aa-6a53-4f4b-a063-e580c3665bb7 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:18.392231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8c3aa-6a53-4f4b-a063-e580c3665bb7 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:18.450221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5c796-d0ad-4da6-9bce-28d94b1c0f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:18.451065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5c796-d0ad-4da6-9bce-28d94b1c0f02 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:18.451355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5c796-d0ad-4da6-9bce-28d94b1c0f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:18.451948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5c796-d0ad-4da6-9bce-28d94b1c0f02 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:18.452376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5c796-d0ad-4da6-9bce-28d94b1c0f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:18.454359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5c796-d0ad-4da6-9bce-28d94b1c0f02 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:18.581767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d602f32-69d8-4ac1-8b87-33428780ece9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:18.581992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1123/2272] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:38:18 2022] POST /identity/v3/users => generated 310 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:18.602257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d602f32-69d8-4ac1-8b87-33428780ece9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:18.608833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bd002-7787-42a2-aa4a-941e64aed288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:18.609448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bd002-7787-42a2-aa4a-941e64aed288 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:18.609816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bd002-7787-42a2-aa4a-941e64aed288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:18.610170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bd002-7787-42a2-aa4a-941e64aed288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:18.611308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bd002-7787-42a2-aa4a-941e64aed288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:18.613571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bd002-7787-42a2-aa4a-941e64aed288 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:18.635601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1150/2273] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:18.641271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5c796-d0ad-4da6-9bce-28d94b1c0f02 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:18.648087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:18.652729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5c796-d0ad-4da6-9bce-28d94b1c0f02 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:18.658079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f53c0-d778-4e65-954c-59c76333fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:18.659062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f53c0-d778-4e65-954c-59c76333fc24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:18.659418 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f53c0-d778-4e65-954c-59c76333fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:18.659738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f53c0-d778-4e65-954c-59c76333fc24 admin admin] PATH_INFO: `/v3/projects/e30401e751004586b756ce20caed5c73/users/0be94c84853f4cd5945b5b90f2678a34/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:18.676127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f53c0-d778-4e65-954c-59c76333fc24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30401e751004586b756ce20caed5c73, user_id=0be94c84853f4cd5945b5b90f2678a34, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:18.678813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f53c0-d778-4e65-954c-59c76333fc24 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:18.787163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1124/2274] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:18 2022] PUT /identity/v3/projects/e30401e751004586b756ce20caed5c73/users/0be94c84853f4cd5945b5b90f2678a34/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:18.792107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bd002-7787-42a2-aa4a-941e64aed288 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:18.802575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bd002-7787-42a2-aa4a-941e64aed288 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:18.863625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982daf1-627e-4fa6-b9a6-2b9a163b5567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:18.864762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982daf1-627e-4fa6-b9a6-2b9a163b5567 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:18.865364 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982daf1-627e-4fa6-b9a6-2b9a163b5567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:18.865592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982daf1-627e-4fa6-b9a6-2b9a163b5567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:18.866316 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982daf1-627e-4fa6-b9a6-2b9a163b5567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:18.867983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982daf1-627e-4fa6-b9a6-2b9a163b5567 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:18.879710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1151/2275] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:18.883943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f53c0-d778-4e65-954c-59c76333fc24 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:18.896270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f53c0-d778-4e65-954c-59c76333fc24 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:18.910075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a44da-fb92-4bc8-9d9c-69b715844a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:18.910075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a44da-fb92-4bc8-9d9c-69b715844a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:18.910075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a44da-fb92-4bc8-9d9c-69b715844a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:18.910075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a44da-fb92-4bc8-9d9c-69b715844a00 admin admin] PATH_INFO: `/v3/projects/e30401e751004586b756ce20caed5c73/users/0be94c84853f4cd5945b5b90f2678a34/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:18.910454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a44da-fb92-4bc8-9d9c-69b715844a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30401e751004586b756ce20caed5c73, user_id=0be94c84853f4cd5945b5b90f2678a34, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:18.910454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a44da-fb92-4bc8-9d9c-69b715844a00 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:18.918982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1125/2276] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:18 2022] PUT /identity/v3/projects/e30401e751004586b756ce20caed5c73/users/0be94c84853f4cd5945b5b90f2678a34/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:38:18.929110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d8c02-2500-4233-ab7d-91599fdcfe03 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:18.932589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d8c02-2500-4233-ab7d-91599fdcfe03 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:18.932589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d8c02-2500-4233-ab7d-91599fdcfe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:18.956849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-478d8c02-2500-4233-ab7d-91599fdcfe03 None None] MFA Rules not processed for user `0be94c84853f4cd5945b5b90f2678a34`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:18.958068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-478d8c02-2500-4233-ab7d-91599fdcfe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:19.026532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1152/2277] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:18 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:19.037320 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a44da-fb92-4bc8-9d9c-69b715844a00 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:19.039030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f34a44da-fb92-4bc8-9d9c-69b715844a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:19.118139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a44da-fb92-4bc8-9d9c-69b715844a00 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:19.123132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ce79a-1b89-480f-b035-2989b182bb1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:19.124119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ce79a-1b89-480f-b035-2989b182bb1d service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:19.126450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ce79a-1b89-480f-b035-2989b182bb1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:19.126450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ce79a-1b89-480f-b035-2989b182bb1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:19.137334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ce79a-1b89-480f-b035-2989b182bb1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:19.138852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ce79a-1b89-480f-b035-2989b182bb1d service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:19.148936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1126/2278] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:19 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:19.664318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d8c02-2500-4233-ab7d-91599fdcfe03 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:19.666729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-478d8c02-2500-4233-ab7d-91599fdcfe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:19.678363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d8c02-2500-4233-ab7d-91599fdcfe03 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:19.681905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc85c94-86c6-40c1-a728-3e56f3a27258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:19.682580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc85c94-86c6-40c1-a728-3e56f3a27258 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:19.682857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc85c94-86c6-40c1-a728-3e56f3a27258 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:19.683143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc85c94-86c6-40c1-a728-3e56f3a27258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:19.726946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:19.742130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc85c94-86c6-40c1-a728-3e56f3a27258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:19.743609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc85c94-86c6-40c1-a728-3e56f3a27258 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:19.753183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1153/2279] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:19 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:29.144607 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:29.923252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=8 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:32.133152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ce79a-1b89-480f-b035-2989b182bb1d service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:32.134599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-df1ce79a-1b89-480f-b035-2989b182bb1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:32.217264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ce79a-1b89-480f-b035-2989b182bb1d service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:32.222942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:32.223901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:32.224182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:32.224457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] PATH_INFO: `/v3/users/50bd90c469d84beaaffc73cdbf413219` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:32.225886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] RBAC: Authorizing `identity:delete_user(user_id=50bd90c469d84beaaffc73cdbf413219)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:32.228589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:32.420183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50bd90c469d84beaaffc73cdbf413219'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:32.420839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50bd90c469d84beaaffc73cdbf413219'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:32.427598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50bd90c469d84beaaffc73cdbf413219'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:32.428250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50bd90c469d84beaaffc73cdbf413219'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:32.439833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50bd90c469d84beaaffc73cdbf413219'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:32.515323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1127/2280] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:38:32 2022] DELETE /identity/v3/users/50bd90c469d84beaaffc73cdbf413219 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:32.517163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360b645-2ce8-4e7c-8dba-48fab95ade03 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:32.517642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360b645-2ce8-4e7c-8dba-48fab95ade03 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:32.517783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360b645-2ce8-4e7c-8dba-48fab95ade03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:32.535586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a360b645-2ce8-4e7c-8dba-48fab95ade03 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:32.536738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a360b645-2ce8-4e7c-8dba-48fab95ade03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:32.595828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1154/2281] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:32.605937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:32.607465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:32.698716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddcbf3-0cc7-4dae-9873-3f0d2f25f4af admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:32.703250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834a18a-e75e-4c65-a559-5dd45deb78e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:32.703972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834a18a-e75e-4c65-a559-5dd45deb78e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:32.704260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834a18a-e75e-4c65-a559-5dd45deb78e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:32.704538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834a18a-e75e-4c65-a559-5dd45deb78e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:32.715507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834a18a-e75e-4c65-a559-5dd45deb78e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:32.717078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834a18a-e75e-4c65-a559-5dd45deb78e1 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:32.727322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1128/2282] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:33.206978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360b645-2ce8-4e7c-8dba-48fab95ade03 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:33.212671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a360b645-2ce8-4e7c-8dba-48fab95ade03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:33.227193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360b645-2ce8-4e7c-8dba-48fab95ade03 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:33.236799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:33.237944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:33.238303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:33.238639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] PATH_INFO: `/v3/projects/9b392ccdc22844bd9cbfc48ffe81ba6b` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:33.239925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b392ccdc22844bd9cbfc48ffe81ba6b)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:33.241999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:33.347273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b392ccdc22844bd9cbfc48ffe81ba6b', 'request_id': 'req-725f14c4-b0be-4f53-b535-0e53459f6839'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:33.409644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b392ccdc22844bd9cbfc48ffe81ba6b', 'request_id': 'req-725f14c4-b0be-4f53-b535-0e53459f6839'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:33.417381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b392ccdc22844bd9cbfc48ffe81ba6b', 'request_id': 'req-725f14c4-b0be-4f53-b535-0e53459f6839'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:33.428948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] The token cache is being invalidate because project 9b392ccdc22844bd9cbfc48ffe81ba6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:38:33.429233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:33.429866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-725f14c4-b0be-4f53-b535-0e53459f6839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:33.432148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1155/2283] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:38:33 2022] DELETE /identity/v3/projects/9b392ccdc22844bd9cbfc48ffe81ba6b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:33.959121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834a18a-e75e-4c65-a559-5dd45deb78e1 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:33.961921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6834a18a-e75e-4c65-a559-5dd45deb78e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:33.968747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:33.975704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834a18a-e75e-4c65-a559-5dd45deb78e1 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:34.051675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:34.053004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:34.053528 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:34.054018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] PATH_INFO: `/v3/users/2f53544224d343a88ca7d2371f4fb1bb` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:34.055476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f53544224d343a88ca7d2371f4fb1bb)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:34.057610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:34.273269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f53544224d343a88ca7d2371f4fb1bb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.274242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f53544224d343a88ca7d2371f4fb1bb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.283745 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f53544224d343a88ca7d2371f4fb1bb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.285623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f53544224d343a88ca7d2371f4fb1bb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.304998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f53544224d343a88ca7d2371f4fb1bb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.425635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1129/2284] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:38:33 2022] DELETE /identity/v3/users/2f53544224d343a88ca7d2371f4fb1bb => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:34.428045 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e1a6-3df6-4a3d-bad4-c508616bb994 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:34.428586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e1a6-3df6-4a3d-bad4-c508616bb994 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:34.428995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e1a6-3df6-4a3d-bad4-c508616bb994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:34.451901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-be12e1a6-3df6-4a3d-bad4-c508616bb994 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:34.453675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-be12e1a6-3df6-4a3d-bad4-c508616bb994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:34.546517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1156/2285] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:34.557667 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:34.559173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:34.620467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03523cd9-3460-4446-8da7-f45dc90ec6a2 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:34.624286 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394346-bd2f-4885-9431-6d49200a8229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:34.625027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394346-bd2f-4885-9431-6d49200a8229 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:34.625307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394346-bd2f-4885-9431-6d49200a8229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:34.625610 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4394346-bd2f-4885-9431-6d49200a8229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:34.639420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4394346-bd2f-4885-9431-6d49200a8229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:34.639862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4394346-bd2f-4885-9431-6d49200a8229 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:34.654492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9156f6c-cb80-4389-b12e-e81422c5449a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:34.662680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9156f6c-cb80-4389-b12e-e81422c5449a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:34.662680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9156f6c-cb80-4389-b12e-e81422c5449a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:34.662876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1130/2286] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:34.686323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394346-bd2f-4885-9431-6d49200a8229 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:34.687469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f4394346-bd2f-4885-9431-6d49200a8229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:34.702473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4394346-bd2f-4885-9431-6d49200a8229 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:34.703439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c9156f6c-cb80-4389-b12e-e81422c5449a None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:34.705914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c9156f6c-cb80-4389-b12e-e81422c5449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:34.709038 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:34.710397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:34.710805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:34.711208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] PATH_INFO: `/v3/users/3568fe48ba0c46739d8bccd988bacaba` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:34.713148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3568fe48ba0c46739d8bccd988bacaba)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:34.716967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:34.721978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:34.747806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:34.780484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1157/2287] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:38:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:34.829527 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9156f6c-cb80-4389-b12e-e81422c5449a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:34.830540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c9156f6c-cb80-4389-b12e-e81422c5449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:34.920793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3568fe48ba0c46739d8bccd988bacaba'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.921879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3568fe48ba0c46739d8bccd988bacaba'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.936357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9156f6c-cb80-4389-b12e-e81422c5449a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:34.937960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3568fe48ba0c46739d8bccd988bacaba'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.938825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3568fe48ba0c46739d8bccd988bacaba'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.942469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1a897-674b-425f-9800-b2b724fcb67d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:34.947062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1a897-674b-425f-9800-b2b724fcb67d service swift] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:34.947062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1a897-674b-425f-9800-b2b724fcb67d service swift] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:34.947062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1a897-674b-425f-9800-b2b724fcb67d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:34.948438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fbb1a897-674b-425f-9800-b2b724fcb67d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:34.953360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3568fe48ba0c46739d8bccd988bacaba'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:34.962528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1a897-674b-425f-9800-b2b724fcb67d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:34.964798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1a897-674b-425f-9800-b2b724fcb67d service swift] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:34.976553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1158/2288] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:38:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 09:38:35.016036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1131/2289] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:38:34 2022] DELETE /identity/v3/users/3568fe48ba0c46739d8bccd988bacaba => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:35.022043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab3b6f-e39b-4c07-8fd8-e5dfbf216a72 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:35.022472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab3b6f-e39b-4c07-8fd8-e5dfbf216a72 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:35.022857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab3b6f-e39b-4c07-8fd8-e5dfbf216a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:35.054514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c6ab3b6f-e39b-4c07-8fd8-e5dfbf216a72 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:35.056629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c6ab3b6f-e39b-4c07-8fd8-e5dfbf216a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:35.068921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:35.077378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:35.144171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1159/2290] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:35.155853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab3b6f-e39b-4c07-8fd8-e5dfbf216a72 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:35.157626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c6ab3b6f-e39b-4c07-8fd8-e5dfbf216a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:35.249467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b05ae-6163-4426-9a01-06efcfdb0059 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:35.259318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:35.260411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:35.260784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:35.262095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] PATH_INFO: `/v3/projects/b9185bb22ed4471aaca5af5422e378c0` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:35.279347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9185bb22ed4471aaca5af5422e378c0)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:35.281348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:35.286556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab3b6f-e39b-4c07-8fd8-e5dfbf216a72 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:35.296174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5576487-ae9c-42b9-8ac9-d731a5ed6d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:35.296855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5576487-ae9c-42b9-8ac9-d731a5ed6d2f service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:35.297147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5576487-ae9c-42b9-8ac9-d731a5ed6d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:35.297588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5576487-ae9c-42b9-8ac9-d731a5ed6d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:35.311460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:35.331448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5576487-ae9c-42b9-8ac9-d731a5ed6d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:35.333022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5576487-ae9c-42b9-8ac9-d731a5ed6d2f service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:35.354907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1160/2291] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:38:35.432135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9185bb22ed4471aaca5af5422e378c0', 'request_id': 'req-adbe7c02-4b38-4d56-8883-9ffd93bdb396'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:35.441686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9185bb22ed4471aaca5af5422e378c0', 'request_id': 'req-adbe7c02-4b38-4d56-8883-9ffd93bdb396'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:35.450699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9185bb22ed4471aaca5af5422e378c0', 'request_id': 'req-adbe7c02-4b38-4d56-8883-9ffd93bdb396'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:35.531803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] The token cache is being invalidate because project b9185bb22ed4471aaca5af5422e378c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:38:35.532346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:35.533195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-adbe7c02-4b38-4d56-8883-9ffd93bdb396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:35.536251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1132/2292] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:38:35 2022] DELETE /identity/v3/projects/b9185bb22ed4471aaca5af5422e378c0 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:38:35.550053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17367ec1-de1b-4931-b5ca-7edecc46ebae None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:35.551496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17367ec1-de1b-4931-b5ca-7edecc46ebae None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:35.551954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17367ec1-de1b-4931-b5ca-7edecc46ebae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:35.665492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-17367ec1-de1b-4931-b5ca-7edecc46ebae None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:35.665492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-17367ec1-de1b-4931-b5ca-7edecc46ebae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:35.813744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1161/2293] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:35.823658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5870a7-e096-41b5-8142-eb2138523424 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:35.827571 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5870a7-e096-41b5-8142-eb2138523424 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:35.827571 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5870a7-e096-41b5-8142-eb2138523424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:35.853978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9c5870a7-e096-41b5-8142-eb2138523424 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:35.855383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9c5870a7-e096-41b5-8142-eb2138523424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:35.867047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1133/2294] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:35.872392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17367ec1-de1b-4931-b5ca-7edecc46ebae None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:35.885640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17367ec1-de1b-4931-b5ca-7edecc46ebae None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:35.906412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:35.907247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:35.918768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:35.918768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:35.918768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:38:35.918768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:35.918768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:35.926369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1162/2295] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:38:35.933268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5870a7-e096-41b5-8142-eb2138523424 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:35.944119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5870a7-e096-41b5-8142-eb2138523424 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:35.954235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d8447-48a3-4b9f-b8bc-9e66ccab4cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:35.955248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d8447-48a3-4b9f-b8bc-9e66ccab4cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:35.955773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d8447-48a3-4b9f-b8bc-9e66ccab4cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:35.956848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d8447-48a3-4b9f-b8bc-9e66ccab4cd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:35.957931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d8447-48a3-4b9f-b8bc-9e66ccab4cd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:35.959939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d8447-48a3-4b9f-b8bc-9e66ccab4cd9 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:36.006161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:36.012751 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:36.028504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ad618-56a6-4c37-9088-330c68ec1b1e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:36.032556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:36.035611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:36.036622 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:36.039856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:36.039856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] PATH_INFO: `/v3/projects/6ca0a71cf4c640ce88e4d686aa1c91b1` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:36.039856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca0a71cf4c640ce88e4d686aa1c91b1)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:36.040752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:36.050995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1134/2296] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:38:35 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:36.054939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d8447-48a3-4b9f-b8bc-9e66ccab4cd9 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:36.142615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d8447-48a3-4b9f-b8bc-9e66ccab4cd9 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:36.147664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916ec3-0b31-4024-8f3f-015a3f2e7646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:36.148341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916ec3-0b31-4024-8f3f-015a3f2e7646 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:36.148619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916ec3-0b31-4024-8f3f-015a3f2e7646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:36.148877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916ec3-0b31-4024-8f3f-015a3f2e7646 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:36.149863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46916ec3-0b31-4024-8f3f-015a3f2e7646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:36.151912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46916ec3-0b31-4024-8f3f-015a3f2e7646 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:36.231201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca0a71cf4c640ce88e4d686aa1c91b1', 'request_id': 'req-372cdb76-7f67-4453-a062-4ac529bd2a87'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:36.255769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1135/2297] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:38:36 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:36.256699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916ec3-0b31-4024-8f3f-015a3f2e7646 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:36.273896 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916ec3-0b31-4024-8f3f-015a3f2e7646 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:36.312311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca0a71cf4c640ce88e4d686aa1c91b1', 'request_id': 'req-372cdb76-7f67-4453-a062-4ac529bd2a87'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:36.335995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca0a71cf4c640ce88e4d686aa1c91b1', 'request_id': 'req-372cdb76-7f67-4453-a062-4ac529bd2a87'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:36.432912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] The token cache is being invalidate because project 6ca0a71cf4c640ce88e4d686aa1c91b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:38:36.432912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:36.434188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:36.436397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1163/2298] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:38:36 2022] DELETE /identity/v3/projects/6ca0a71cf4c640ce88e4d686aa1c91b1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:36.445332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93959cef-42ac-4460-aa25-325a35b249bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:36.447045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93959cef-42ac-4460-aa25-325a35b249bc admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:36.447603 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93959cef-42ac-4460-aa25-325a35b249bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:36.447852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:36.448424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93959cef-42ac-4460-aa25-325a35b249bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:36.449491 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:36.449731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93959cef-42ac-4460-aa25-325a35b249bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:36.452477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93959cef-42ac-4460-aa25-325a35b249bc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:36.468589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1136/2299] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:38:36.475031 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93959cef-42ac-4460-aa25-325a35b249bc admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:36.477731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cdb76-7f67-4453-a062-4ac529bd2a87 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:36.481410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-93959cef-42ac-4460-aa25-325a35b249bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:36.485787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:36.487087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:36.487087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:36.487519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] PATH_INFO: `/v3/users/0be94c84853f4cd5945b5b90f2678a34` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:36.488955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be94c84853f4cd5945b5b90f2678a34)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:36.491012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:36.504017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93959cef-42ac-4460-aa25-325a35b249bc admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:36.513014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c7468-effa-403d-ba01-943db2845fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:36.513585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c7468-effa-403d-ba01-943db2845fab admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:36.513860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c7468-effa-403d-ba01-943db2845fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:36.514139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c7468-effa-403d-ba01-943db2845fab admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:36.514876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c7468-effa-403d-ba01-943db2845fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:36.516617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c7468-effa-403d-ba01-943db2845fab admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:36.532400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1137/2300] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:36.536468 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c7468-effa-403d-ba01-943db2845fab admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:36.547809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c7468-effa-403d-ba01-943db2845fab admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:36.553307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977641b-5d22-4a1e-b36c-96880b556f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:36.554344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977641b-5d22-4a1e-b36c-96880b556f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:36.554628 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977641b-5d22-4a1e-b36c-96880b556f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:36.554907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977641b-5d22-4a1e-b36c-96880b556f2e admin admin] PATH_INFO: `/v3/projects/55f89ed92a124e2c8a274975bc0935ba/users/5778103c646049f5944934e775cba9da/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:36.574718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977641b-5d22-4a1e-b36c-96880b556f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f89ed92a124e2c8a274975bc0935ba, user_id=5778103c646049f5944934e775cba9da, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:36.577627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977641b-5d22-4a1e-b36c-96880b556f2e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:36.974151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1138/2301] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:36 2022] PUT /identity/v3/projects/55f89ed92a124e2c8a274975bc0935ba/users/5778103c646049f5944934e775cba9da/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:38:36.974793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977641b-5d22-4a1e-b36c-96880b556f2e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:36.978993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e977641b-5d22-4a1e-b36c-96880b556f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:37.079507 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:37.081514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be94c84853f4cd5945b5b90f2678a34'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.086514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:37.106042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977641b-5d22-4a1e-b36c-96880b556f2e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:37.111515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:37.112449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:37.112738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:37.113066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] PATH_INFO: `/v3/users/eb462af0f24c4ddebe605d8ed636b8ea` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:37.114900 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb462af0f24c4ddebe605d8ed636b8ea)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:37.116776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:37.192470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be94c84853f4cd5945b5b90f2678a34'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.193459 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be94c84853f4cd5945b5b90f2678a34'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.202706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be94c84853f4cd5945b5b90f2678a34'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.206300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d2811fc0-18f0-4d69-8e58-baf24ff271b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be94c84853f4cd5945b5b90f2678a34'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.242687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1164/2302] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:38:36 2022] DELETE /identity/v3/users/0be94c84853f4cd5945b5b90f2678a34 => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:37.244170 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de4cce-090c-479f-af63-245ec7f4df3e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:37.244458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de4cce-090c-479f-af63-245ec7f4df3e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:37.244728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de4cce-090c-479f-af63-245ec7f4df3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:37.297722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e9de4cce-090c-479f-af63-245ec7f4df3e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:37.299468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e9de4cce-090c-479f-af63-245ec7f4df3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:37.385270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb462af0f24c4ddebe605d8ed636b8ea'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.386673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb462af0f24c4ddebe605d8ed636b8ea'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.402108 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb462af0f24c4ddebe605d8ed636b8ea'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.402426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb462af0f24c4ddebe605d8ed636b8ea'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.426640 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1165/2303] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:37.429686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a9bfc56e-921f-41d7-84ee-b661727ed42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb462af0f24c4ddebe605d8ed636b8ea'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:37.439272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de4cce-090c-479f-af63-245ec7f4df3e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:37.440785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e9de4cce-090c-479f-af63-245ec7f4df3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:37.521932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1139/2304] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:38:36 2022] DELETE /identity/v3/users/eb462af0f24c4ddebe605d8ed636b8ea => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:37.523514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de4cce-090c-479f-af63-245ec7f4df3e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:37.526519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fba4c2-9b55-4f2b-a6db-29a4084c61ea None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:37.526740 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fba4c2-9b55-4f2b-a6db-29a4084c61ea None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:37.527023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fba4c2-9b55-4f2b-a6db-29a4084c61ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:37.612075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-c1fba4c2-9b55-4f2b-a6db-29a4084c61ea None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:37.612075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c1fba4c2-9b55-4f2b-a6db-29a4084c61ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:37.617796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564d33d-2d00-404b-8e56-0209c02988c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:37.619132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564d33d-2d00-404b-8e56-0209c02988c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:37.620236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564d33d-2d00-404b-8e56-0209c02988c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:37.621146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564d33d-2d00-404b-8e56-0209c02988c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:37.692727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:37.713949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1140/2305] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:37.722481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564d33d-2d00-404b-8e56-0209c02988c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:37.726645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564d33d-2d00-404b-8e56-0209c02988c7 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:37.736172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fba4c2-9b55-4f2b-a6db-29a4084c61ea None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:37.737792 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c1fba4c2-9b55-4f2b-a6db-29a4084c61ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:37.740425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1166/2306] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:37.762004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564d33d-2d00-404b-8e56-0209c02988c7 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:37.765774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-0564d33d-2d00-404b-8e56-0209c02988c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:37.777687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fba4c2-9b55-4f2b-a6db-29a4084c61ea None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:37.785778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564d33d-2d00-404b-8e56-0209c02988c7 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:37.789264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa31e60-dcea-49ec-bad4-909aa5fa3fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:37.791091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa31e60-dcea-49ec-bad4-909aa5fa3fdc service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:37.791730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa31e60-dcea-49ec-bad4-909aa5fa3fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:37.792049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa31e60-dcea-49ec-bad4-909aa5fa3fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:37.793384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:37.794760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:37.795224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:37.795669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] PATH_INFO: `/v3/projects/e30401e751004586b756ce20caed5c73` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:37.808191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e30401e751004586b756ce20caed5c73)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:37.808921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa31e60-dcea-49ec-bad4-909aa5fa3fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:37.810711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa31e60-dcea-49ec-bad4-909aa5fa3fdc service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:37.815254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:37.823751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1141/2307] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:37.970546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e30401e751004586b756ce20caed5c73', 'request_id': 'req-c8dea936-28bc-4a48-9a4b-5a777c955336'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.038272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e30401e751004586b756ce20caed5c73', 'request_id': 'req-c8dea936-28bc-4a48-9a4b-5a777c955336'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.049496 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e30401e751004586b756ce20caed5c73', 'request_id': 'req-c8dea936-28bc-4a48-9a4b-5a777c955336'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.064043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] The token cache is being invalidate because project e30401e751004586b756ce20caed5c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:38:38.064589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.065617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c8dea936-28bc-4a48-9a4b-5a777c955336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.067380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1167/2308] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:38:37 2022] DELETE /identity/v3/projects/e30401e751004586b756ce20caed5c73 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:38.073620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66728cf5-da58-4f44-ba59-63e9dfd4b577 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.073620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66728cf5-da58-4f44-ba59-63e9dfd4b577 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.073620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66728cf5-da58-4f44-ba59-63e9dfd4b577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.125717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-66728cf5-da58-4f44-ba59-63e9dfd4b577 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:38.128435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-66728cf5-da58-4f44-ba59-63e9dfd4b577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:38.226934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:38.248308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1142/2309] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:38.253090 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd479ddc-6f47-4dba-b87c-8227f01763e3 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.253608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd479ddc-6f47-4dba-b87c-8227f01763e3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.253775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd479ddc-6f47-4dba-b87c-8227f01763e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.284279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-fd479ddc-6f47-4dba-b87c-8227f01763e3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:38.286068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fd479ddc-6f47-4dba-b87c-8227f01763e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:38.297206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1168/2310] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:38.301490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66728cf5-da58-4f44-ba59-63e9dfd4b577 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:38.311829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66728cf5-da58-4f44-ba59-63e9dfd4b577 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:38.317049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:38.317737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.317999 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.318276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.319073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:38:38.319393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:38.321051 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:38.342972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1143/2311] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:38:38.347337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd479ddc-6f47-4dba-b87c-8227f01763e3 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:38.358402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd479ddc-6f47-4dba-b87c-8227f01763e3 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:38.368118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf29125-0a22-489a-b3da-46583d863135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:38.368904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf29125-0a22-489a-b3da-46583d863135 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.370503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf29125-0a22-489a-b3da-46583d863135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.371189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf29125-0a22-489a-b3da-46583d863135 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.375577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf29125-0a22-489a-b3da-46583d863135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:38.377615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf29125-0a22-489a-b3da-46583d863135 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:38.429181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:38.431803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:38.468823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2b861-df36-4bc0-b14a-9be875a4b694 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:38.491121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1169/2312] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:38:38 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:38.492144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:38.493273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.493712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.494671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] PATH_INFO: `/v3/projects/d6a8fa7f6bdb43c4941aeb6a22a6c1bc` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.495615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf29125-0a22-489a-b3da-46583d863135 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:38.497041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a8fa7f6bdb43c4941aeb6a22a6c1bc)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:38.502202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:38.607036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf29125-0a22-489a-b3da-46583d863135 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:38.616267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0265f7-795a-449c-b567-cd07de9e90f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:38.617067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0265f7-795a-449c-b567-cd07de9e90f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.618350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0265f7-795a-449c-b567-cd07de9e90f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.618704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0265f7-795a-449c-b567-cd07de9e90f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.619877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0265f7-795a-449c-b567-cd07de9e90f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:38.622092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0265f7-795a-449c-b567-cd07de9e90f2 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:38.646115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a8fa7f6bdb43c4941aeb6a22a6c1bc', 'request_id': 'req-982076e2-dbaa-4515-8e1b-80d3581a57c3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.661472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a8fa7f6bdb43c4941aeb6a22a6c1bc', 'request_id': 'req-982076e2-dbaa-4515-8e1b-80d3581a57c3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.672585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a8fa7f6bdb43c4941aeb6a22a6c1bc', 'request_id': 'req-982076e2-dbaa-4515-8e1b-80d3581a57c3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.709896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1170/2313] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:38:38 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:38:38.714421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0265f7-795a-449c-b567-cd07de9e90f2 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:38.730609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0265f7-795a-449c-b567-cd07de9e90f2 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:38.754119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] The token cache is being invalidate because project d6a8fa7f6bdb43c4941aeb6a22a6c1bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:38:38.759735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.759735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-982076e2-dbaa-4515-8e1b-80d3581a57c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:38:38.759735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1144/2314] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:38:38 2022] DELETE /identity/v3/projects/d6a8fa7f6bdb43c4941aeb6a22a6c1bc => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:38:38.762542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaa82d-7504-4394-8672-f91f633655bc None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.762833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaa82d-7504-4394-8672-f91f633655bc None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.763111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaa82d-7504-4394-8672-f91f633655bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.772802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:38.792410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9cdaa82d-7504-4394-8672-f91f633655bc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:38.794987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9cdaa82d-7504-4394-8672-f91f633655bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:38.861102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c567ed4-0dfe-4e40-a16e-ad0821fb8407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:38.862065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c567ed4-0dfe-4e40-a16e-ad0821fb8407 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.862210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c567ed4-0dfe-4e40-a16e-ad0821fb8407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.862501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c567ed4-0dfe-4e40-a16e-ad0821fb8407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.863329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c567ed4-0dfe-4e40-a16e-ad0821fb8407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:38.865354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c567ed4-0dfe-4e40-a16e-ad0821fb8407 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:38.879572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1171/2315] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:38.880940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c567ed4-0dfe-4e40-a16e-ad0821fb8407 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:38.884643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5c567ed4-0dfe-4e40-a16e-ad0821fb8407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:38.911110 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c567ed4-0dfe-4e40-a16e-ad0821fb8407 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:38.913902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1145/2316] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:38.922355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f15b1d-a7fb-4635-81df-6d36a18470f5 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.922889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f15b1d-a7fb-4635-81df-6d36a18470f5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.923350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f15b1d-a7fb-4635-81df-6d36a18470f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.954894 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-71f15b1d-a7fb-4635-81df-6d36a18470f5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:38.955648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ed361-3d2c-4ad1-89f4-74f3675fe6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:38.956522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ed361-3d2c-4ad1-89f4-74f3675fe6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:38.957735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-71f15b1d-a7fb-4635-81df-6d36a18470f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:38.957934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ed361-3d2c-4ad1-89f4-74f3675fe6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:38.958230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ed361-3d2c-4ad1-89f4-74f3675fe6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:38.961127 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ed361-3d2c-4ad1-89f4-74f3675fe6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:38.962851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ed361-3d2c-4ad1-89f4-74f3675fe6d7 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:38.972442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1146/2317] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:38.977015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f15b1d-a7fb-4635-81df-6d36a18470f5 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:38.985548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1172/2318] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:38.992062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ed361-3d2c-4ad1-89f4-74f3675fe6d7 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:39.002859 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ed361-3d2c-4ad1-89f4-74f3675fe6d7 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:39.007758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc618f-aeae-4232-b8e9-55ab88496983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:39.009864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc618f-aeae-4232-b8e9-55ab88496983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:39.009864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc618f-aeae-4232-b8e9-55ab88496983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:39.009864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc618f-aeae-4232-b8e9-55ab88496983 admin admin] PATH_INFO: `/v3/projects/604921be35cc4c258933114a5535800c/users/09f242a8f51545199a159f6992ee74f1/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:39.010438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f15b1d-a7fb-4635-81df-6d36a18470f5 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:39.019515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b46c5-dd05-4303-9293-3e256e1f6e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:39.020521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b46c5-dd05-4303-9293-3e256e1f6e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:39.020901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b46c5-dd05-4303-9293-3e256e1f6e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:39.021244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b46c5-dd05-4303-9293-3e256e1f6e07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:39.022153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b46c5-dd05-4303-9293-3e256e1f6e07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:38:39.022528 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b46c5-dd05-4303-9293-3e256e1f6e07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:39.029954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b46c5-dd05-4303-9293-3e256e1f6e07 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:39.044696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1147/2319] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:38:39.047037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc618f-aeae-4232-b8e9-55ab88496983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=604921be35cc4c258933114a5535800c, user_id=09f242a8f51545199a159f6992ee74f1, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:39.048482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b46c5-dd05-4303-9293-3e256e1f6e07 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:39.056498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc618f-aeae-4232-b8e9-55ab88496983 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:39.089123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b46c5-dd05-4303-9293-3e256e1f6e07 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:39.095069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d399f3c-bd01-4741-ba3d-1fcac2a1de0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:39.095901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d399f3c-bd01-4741-ba3d-1fcac2a1de0d admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:39.096350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d399f3c-bd01-4741-ba3d-1fcac2a1de0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:39.096669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d399f3c-bd01-4741-ba3d-1fcac2a1de0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:39.097485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d399f3c-bd01-4741-ba3d-1fcac2a1de0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:39.100037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d399f3c-bd01-4741-ba3d-1fcac2a1de0d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:39.165447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1173/2320] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:38 2022] PUT /identity/v3/projects/604921be35cc4c258933114a5535800c/users/09f242a8f51545199a159f6992ee74f1/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:39.176302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8cf28-8dcd-48a0-b80d-c23738ff1fb5 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:39.176596 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8cf28-8dcd-48a0-b80d-c23738ff1fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:39.176892 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8cf28-8dcd-48a0-b80d-c23738ff1fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:39.209615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2dc8cf28-8dcd-48a0-b80d-c23738ff1fb5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:39.211495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2dc8cf28-8dcd-48a0-b80d-c23738ff1fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:39.255988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1148/2321] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:38:39 2022] POST /identity/v3/projects => generated 375 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:39.257621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d399f3c-bd01-4741-ba3d-1fcac2a1de0d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:39.366776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d399f3c-bd01-4741-ba3d-1fcac2a1de0d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:39.374682 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ccde0-7336-4d5d-a2fa-95b037cfb50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:39.375390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ccde0-7336-4d5d-a2fa-95b037cfb50e admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:39.376093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ccde0-7336-4d5d-a2fa-95b037cfb50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:39.409027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ccde0-7336-4d5d-a2fa-95b037cfb50e admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:39.409027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ccde0-7336-4d5d-a2fa-95b037cfb50e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:39.409027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ccde0-7336-4d5d-a2fa-95b037cfb50e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:39.410048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1174/2322] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:38:39.410048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8cf28-8dcd-48a0-b80d-c23738ff1fb5 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:39.410048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2dc8cf28-8dcd-48a0-b80d-c23738ff1fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:39.471097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8cf28-8dcd-48a0-b80d-c23738ff1fb5 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:39.519029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c236f14-46f0-4a21-b47d-c0380c676f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:39.519721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c236f14-46f0-4a21-b47d-c0380c676f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:39.520000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c236f14-46f0-4a21-b47d-c0380c676f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:39.520273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c236f14-46f0-4a21-b47d-c0380c676f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:39.529700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c236f14-46f0-4a21-b47d-c0380c676f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:39.531118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c236f14-46f0-4a21-b47d-c0380c676f05 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:39.541200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1175/2323] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:39.812948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1149/2324] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:38:39 2022] POST /identity/v3/users => generated 288 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:39.814881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c236f14-46f0-4a21-b47d-c0380c676f05 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:39.842424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c236f14-46f0-4a21-b47d-c0380c676f05 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:39.850042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959be4b-338a-4383-bd93-f957361d677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:39.854391 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959be4b-338a-4383-bd93-f957361d677a admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:39.854391 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959be4b-338a-4383-bd93-f957361d677a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:39.854391 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959be4b-338a-4383-bd93-f957361d677a admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:39.854391 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959be4b-338a-4383-bd93-f957361d677a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:39.854919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959be4b-338a-4383-bd93-f957361d677a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:39.869877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1176/2325] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:39.873763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ccde0-7336-4d5d-a2fa-95b037cfb50e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:39.909585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ccde0-7336-4d5d-a2fa-95b037cfb50e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:39.921027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81323570-f6d6-4768-bd17-bdc32de19e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:39.925177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81323570-f6d6-4768-bd17-bdc32de19e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:39.925177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81323570-f6d6-4768-bd17-bdc32de19e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:39.925177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81323570-f6d6-4768-bd17-bdc32de19e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:39.925177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81323570-f6d6-4768-bd17-bdc32de19e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:39.925859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81323570-f6d6-4768-bd17-bdc32de19e78 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:39.945342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1150/2326] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:39.949002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959be4b-338a-4383-bd93-f957361d677a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:39.974971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959be4b-338a-4383-bd93-f957361d677a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:39.980540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35625a5d-6d35-4c2c-ac54-8230eb31624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:39.981582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625a5d-6d35-4c2c-ac54-8230eb31624d admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:39.982198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625a5d-6d35-4c2c-ac54-8230eb31624d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:39.982477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625a5d-6d35-4c2c-ac54-8230eb31624d admin admin] PATH_INFO: `/v3/projects/0eb2a1214a1a4ae5a5211c18c2c39c1a/users/fdf2758dad91482ea5da6bfdb5896313/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:39.998307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35625a5d-6d35-4c2c-ac54-8230eb31624d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb2a1214a1a4ae5a5211c18c2c39c1a, user_id=fdf2758dad91482ea5da6bfdb5896313, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:40.001317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35625a5d-6d35-4c2c-ac54-8230eb31624d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:40.091367 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1177/2327] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:39 2022] PUT /identity/v3/projects/0eb2a1214a1a4ae5a5211c18c2c39c1a/users/fdf2758dad91482ea5da6bfdb5896313/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:40.101970 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25f08e-61cc-4245-9611-993368b6c8c4 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:40.102231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25f08e-61cc-4245-9611-993368b6c8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:40.102543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25f08e-61cc-4245-9611-993368b6c8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:40.170840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-fa25f08e-61cc-4245-9611-993368b6c8c4 None None] MFA Rules not processed for user `fdf2758dad91482ea5da6bfdb5896313`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:40.172141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fa25f08e-61cc-4245-9611-993368b6c8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:40.194442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:40.297595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1151/2328] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:40.307092 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35625a5d-6d35-4c2c-ac54-8230eb31624d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:40.309303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-35625a5d-6d35-4c2c-ac54-8230eb31624d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:40.380974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35625a5d-6d35-4c2c-ac54-8230eb31624d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:40.385526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcae932-faf4-4328-98ba-f07049ddc898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:40.386310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcae932-faf4-4328-98ba-f07049ddc898 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:40.386809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcae932-faf4-4328-98ba-f07049ddc898 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:40.387248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcae932-faf4-4328-98ba-f07049ddc898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:40.397283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcae932-faf4-4328-98ba-f07049ddc898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:40.398846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcae932-faf4-4328-98ba-f07049ddc898 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:40.424419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1178/2329] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:41.049979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:41.948034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3453f-51c1-42d7-920a-cfe369fe5a0c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:41.948551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3453f-51c1-42d7-920a-cfe369fe5a0c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:41.948649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3453f-51c1-42d7-920a-cfe369fe5a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:42.062949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-07f3453f-51c1-42d7-920a-cfe369fe5a0c None None] MFA Rules not processed for user `09f242a8f51545199a159f6992ee74f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:42.065517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-07f3453f-51c1-42d7-920a-cfe369fe5a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:42.242354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1152/2330] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:42.253338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcae932-faf4-4328-98ba-f07049ddc898 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:42.264613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcae932-faf4-4328-98ba-f07049ddc898 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:42.272013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851496e-7135-4b59-a2cc-39ff9dbb62f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:42.272662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851496e-7135-4b59-a2cc-39ff9dbb62f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:42.273019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851496e-7135-4b59-a2cc-39ff9dbb62f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:42.273395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851496e-7135-4b59-a2cc-39ff9dbb62f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:42.295217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851496e-7135-4b59-a2cc-39ff9dbb62f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:42.296787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851496e-7135-4b59-a2cc-39ff9dbb62f7 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:42.340284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1179/2331] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:42.461010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b1c3c-abe2-428e-b775-3a871f2d0ad7 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:42.462261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b1c3c-abe2-428e-b775-3a871f2d0ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:42.462708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b1c3c-abe2-428e-b775-3a871f2d0ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:42.484940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b64b1c3c-abe2-428e-b775-3a871f2d0ad7 None None] MFA Rules not processed for user `fdf2758dad91482ea5da6bfdb5896313`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:42.486623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b64b1c3c-abe2-428e-b775-3a871f2d0ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:42.501360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1153/2332] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:44.387560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567242b4-1a01-462d-9f54-c3c01f78ff11 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:44.388039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567242b4-1a01-462d-9f54-c3c01f78ff11 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:44.388144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567242b4-1a01-462d-9f54-c3c01f78ff11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:44.446996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-567242b4-1a01-462d-9f54-c3c01f78ff11 None None] MFA Rules not processed for user `09f242a8f51545199a159f6992ee74f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:44.466308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-567242b4-1a01-462d-9f54-c3c01f78ff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:44.512133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1180/2333] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:45.187799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d52c8-261f-462b-b6aa-bed423835e70 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:45.188389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d52c8-261f-462b-b6aa-bed423835e70 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:45.188389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d52c8-261f-462b-b6aa-bed423835e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:45.209615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-720d52c8-261f-462b-b6aa-bed423835e70 None None] MFA Rules not processed for user `41ce5cc7ff504d64aac73e48097a4b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:45.211118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-720d52c8-261f-462b-b6aa-bed423835e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:45.239580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:45.356586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1154/2334] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:46.581690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa1dc9-d38a-4cb8-8199-3d1bc4f49ef7 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:46.582258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa1dc9-d38a-4cb8-8199-3d1bc4f49ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:46.582258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa1dc9-d38a-4cb8-8199-3d1bc4f49ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:46.644648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-73aa1dc9-d38a-4cb8-8199-3d1bc4f49ef7 None None] MFA Rules not processed for user `5778103c646049f5944934e775cba9da`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:46.646595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-73aa1dc9-d38a-4cb8-8199-3d1bc4f49ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:46.829532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1181/2335] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:46.840497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d52c8-261f-462b-b6aa-bed423835e70 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:46.844832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-720d52c8-261f-462b-b6aa-bed423835e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:46.961612 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d52c8-261f-462b-b6aa-bed423835e70 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:46.967083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35053d47-bc2f-4123-a72d-dd0f3e716e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:46.968048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35053d47-bc2f-4123-a72d-dd0f3e716e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:46.968506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35053d47-bc2f-4123-a72d-dd0f3e716e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:46.968968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35053d47-bc2f-4123-a72d-dd0f3e716e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:46.969952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35053d47-bc2f-4123-a72d-dd0f3e716e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:46.972128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35053d47-bc2f-4123-a72d-dd0f3e716e1b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:47.100511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa1dc9-d38a-4cb8-8199-3d1bc4f49ef7 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:47.114333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1155/2336] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:38:46 2022] POST /identity/v3/projects => generated 383 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:47.121054 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa1dc9-d38a-4cb8-8199-3d1bc4f49ef7 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:47.144845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:47.221217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201130f2-64e0-4ed9-9ceb-3f8b040e0a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:47.221916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201130f2-64e0-4ed9-9ceb-3f8b040e0a53 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:47.222199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201130f2-64e0-4ed9-9ceb-3f8b040e0a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:47.222476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201130f2-64e0-4ed9-9ceb-3f8b040e0a53 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:47.223173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201130f2-64e0-4ed9-9ceb-3f8b040e0a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:47.225389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201130f2-64e0-4ed9-9ceb-3f8b040e0a53 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:47.326484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1182/2337] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:38:47 2022] POST /identity/v3/users => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:38:47.329116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35053d47-bc2f-4123-a72d-dd0f3e716e1b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:47.348272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35053d47-bc2f-4123-a72d-dd0f3e716e1b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:47.360806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88590808-1965-4354-884e-617ee72aca96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:47.361145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88590808-1965-4354-884e-617ee72aca96 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:47.361739 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88590808-1965-4354-884e-617ee72aca96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:47.362056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88590808-1965-4354-884e-617ee72aca96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:47.363032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88590808-1965-4354-884e-617ee72aca96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:47.365212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88590808-1965-4354-884e-617ee72aca96 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:47.391020 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201130f2-64e0-4ed9-9ceb-3f8b040e0a53 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:47.395763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1156/2338] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:47.409389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201130f2-64e0-4ed9-9ceb-3f8b040e0a53 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:47.415970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03a5c0-d94f-4764-9c4a-f680e257cdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:47.417158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a5c0-d94f-4764-9c4a-f680e257cdb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:47.417565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a5c0-d94f-4764-9c4a-f680e257cdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:47.417940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a5c0-d94f-4764-9c4a-f680e257cdb2 admin admin] PATH_INFO: `/v3/projects/9550c13ccce1494b820141ec0acc768e/users/1dcca7ce110547b8a6bfdf4a84f6f220/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:47.439674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03a5c0-d94f-4764-9c4a-f680e257cdb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9550c13ccce1494b820141ec0acc768e, user_id=1dcca7ce110547b8a6bfdf4a84f6f220, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:47.443887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03a5c0-d94f-4764-9c4a-f680e257cdb2 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:47.565841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1183/2339] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:47 2022] PUT /identity/v3/projects/9550c13ccce1494b820141ec0acc768e/users/1dcca7ce110547b8a6bfdf4a84f6f220/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:38:47.566905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88590808-1965-4354-884e-617ee72aca96 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:47.584903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88590808-1965-4354-884e-617ee72aca96 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:47.679483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b261ae-b205-4cf1-acdf-5fd92b7986b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:47.680201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b261ae-b205-4cf1-acdf-5fd92b7986b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:47.680713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b261ae-b205-4cf1-acdf-5fd92b7986b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:47.681037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b261ae-b205-4cf1-acdf-5fd92b7986b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:47.682878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b261ae-b205-4cf1-acdf-5fd92b7986b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:47.683465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b261ae-b205-4cf1-acdf-5fd92b7986b3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:47.699421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1157/2340] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:38:47.700629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03a5c0-d94f-4764-9c4a-f680e257cdb2 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:47.715857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03a5c0-d94f-4764-9c4a-f680e257cdb2 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:47.730473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a8123-9ea5-46da-a387-fd9f8d9aa8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:47.731395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a8123-9ea5-46da-a387-fd9f8d9aa8a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:47.731709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a8123-9ea5-46da-a387-fd9f8d9aa8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:47.732049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a8123-9ea5-46da-a387-fd9f8d9aa8a6 admin admin] PATH_INFO: `/v3/projects/9550c13ccce1494b820141ec0acc768e/users/1dcca7ce110547b8a6bfdf4a84f6f220/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:47.735107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a8123-9ea5-46da-a387-fd9f8d9aa8a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9550c13ccce1494b820141ec0acc768e, user_id=1dcca7ce110547b8a6bfdf4a84f6f220, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:47.737844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a8123-9ea5-46da-a387-fd9f8d9aa8a6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:47.751571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1184/2341] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:38:47 2022] PUT /identity/v3/projects/9550c13ccce1494b820141ec0acc768e/users/1dcca7ce110547b8a6bfdf4a84f6f220/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:38:56.662595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08328f96-e8ac-447f-8c1e-9db39ee5d6bd None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:56.663075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08328f96-e8ac-447f-8c1e-9db39ee5d6bd None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:56.663461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08328f96-e8ac-447f-8c1e-9db39ee5d6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:56.718888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-08328f96-e8ac-447f-8c1e-9db39ee5d6bd None None] MFA Rules not processed for user `1dcca7ce110547b8a6bfdf4a84f6f220`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:56.720187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-08328f96-e8ac-447f-8c1e-9db39ee5d6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:56.744340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:56.812990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1158/2342] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:38:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:56.825752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a8123-9ea5-46da-a387-fd9f8d9aa8a6 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:56.828772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c12a8123-9ea5-46da-a387-fd9f8d9aa8a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:56.954412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a8123-9ea5-46da-a387-fd9f8d9aa8a6 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:56.959187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f314894-90d8-4cbe-b48c-8348dc3aaaad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e33d1473077247cc97dcfd99b3da5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:56.960681 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f314894-90d8-4cbe-b48c-8348dc3aaaad service glance] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:56.960803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f314894-90d8-4cbe-b48c-8348dc3aaaad service glance] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:56.961025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f314894-90d8-4cbe-b48c-8348dc3aaaad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:57.061409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f314894-90d8-4cbe-b48c-8348dc3aaaad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:57.065066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f314894-90d8-4cbe-b48c-8348dc3aaaad service glance] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:57.094921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1185/2343] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:57.355402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ca0ce-09ba-430c-b7c2-9d0f5e2ea4d5 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:57.355652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ca0ce-09ba-430c-b7c2-9d0f5e2ea4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:57.356510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ca0ce-09ba-430c-b7c2-9d0f5e2ea4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:57.378922 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9b7ca0ce-09ba-430c-b7c2-9d0f5e2ea4d5 None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:57.380309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9b7ca0ce-09ba-430c-b7c2-9d0f5e2ea4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:57.539011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1159/2344] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:38:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:57.562512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f314894-90d8-4cbe-b48c-8348dc3aaaad service glance] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:57.564505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2f314894-90d8-4cbe-b48c-8348dc3aaaad service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:57.625361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:38:57.676741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f314894-90d8-4cbe-b48c-8348dc3aaaad service glance] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:57.682524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e77ee8-8b0c-41d0-b679-259ab7281bcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:57.683201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e77ee8-8b0c-41d0-b679-259ab7281bcb service swift] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:57.683487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e77ee8-8b0c-41d0-b679-259ab7281bcb service swift] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:57.683809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e77ee8-8b0c-41d0-b679-259ab7281bcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:57.693087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e77ee8-8b0c-41d0-b679-259ab7281bcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:57.694705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e77ee8-8b0c-41d0-b679-259ab7281bcb service swift] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:57.705794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1186/2345] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:38:58.887885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d3d10-4fc1-4786-b532-93f9b692fb56 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:58.888494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d3d10-4fc1-4786-b532-93f9b692fb56 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:58.888494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d3d10-4fc1-4786-b532-93f9b692fb56 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:58.889839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1160/2346] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:38:58.902938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a36d1-82b1-46c5-bf3f-01ce4df73b00 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:58.904304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a36d1-82b1-46c5-bf3f-01ce4df73b00 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:58.910111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a36d1-82b1-46c5-bf3f-01ce4df73b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:59.038051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-f15a36d1-82b1-46c5-bf3f-01ce4df73b00 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:59.039805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f15a36d1-82b1-46c5-bf3f-01ce4df73b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:59.266212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1187/2347] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:38:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:38:59.296626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d3d10-4fc1-4786-b532-93f9b692fb56 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:38:59.318019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d3d10-4fc1-4786-b532-93f9b692fb56 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:38:59.325559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5b8eb-aed4-4854-9a50-c0501ee4baa9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e33d1473077247cc97dcfd99b3da5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:38:59.326670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b8eb-aed4-4854-9a50-c0501ee4baa9 service glance] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:59.327115 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b8eb-aed4-4854-9a50-c0501ee4baa9 service glance] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:59.331584 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b8eb-aed4-4854-9a50-c0501ee4baa9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:59.369562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5b8eb-aed4-4854-9a50-c0501ee4baa9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:38:59.371630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5b8eb-aed4-4854-9a50-c0501ee4baa9 service glance] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:38:59.402599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1161/2348] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:38:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:38:59.658485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb0ec5-1804-454c-aa32-ca2ae38cdda5 None None] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:59.662697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb0ec5-1804-454c-aa32-ca2ae38cdda5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:59.662697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb0ec5-1804-454c-aa32-ca2ae38cdda5 None None] PATH_INFO: `/` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:59.662697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1188/2349] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:38:59.674978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33d623-0e94-41b2-a127-2e1fa49597df None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:38:59.675273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33d623-0e94-41b2-a127-2e1fa49597df None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:38:59.675584 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33d623-0e94-41b2-a127-2e1fa49597df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:38:59.723464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7a33d623-0e94-41b2-a127-2e1fa49597df None None] MFA Rules not processed for user `2b87f03cae6c4ca98a82dd5d981ed224`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:38:59.725887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7a33d623-0e94-41b2-a127-2e1fa49597df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:38:59.954847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1162/2350] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:38:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:00.005546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb0ec5-1804-454c-aa32-ca2ae38cdda5 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:00.009694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a2eb0ec5-1804-454c-aa32-ca2ae38cdda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:00.063425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb0ec5-1804-454c-aa32-ca2ae38cdda5 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:00.068398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c3255-d636-45a8-a425-b4280f18a9f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:00.069320 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c3255-d636-45a8-a425-b4280f18a9f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:00.069762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c3255-d636-45a8-a425-b4280f18a9f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:00.070266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c3255-d636-45a8-a425-b4280f18a9f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:00.110869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c3255-d636-45a8-a425-b4280f18a9f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:00.113201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c3255-d636-45a8-a425-b4280f18a9f0 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:00.179729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1189/2351] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:39:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:39:00.528073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d4abf-422a-492e-b084-cf9487967a02 None None] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:00.528406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d4abf-422a-492e-b084-cf9487967a02 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:00.528690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d4abf-422a-492e-b084-cf9487967a02 None None] PATH_INFO: `/` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:00.530095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1163/2352] 10.222.0.64 () {54 vars in 921 bytes} [Mon Jun 27 09:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 09:39:00.550990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15798ec9-4c81-4f15-bb07-6e2770fc5fd3 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:00.550990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15798ec9-4c81-4f15-bb07-6e2770fc5fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:00.550990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15798ec9-4c81-4f15-bb07-6e2770fc5fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:00.597600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-15798ec9-4c81-4f15-bb07-6e2770fc5fd3 None None] MFA Rules not processed for user `19e0fbe0b5814ad1ae95c65a64b632d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:00.598983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-15798ec9-4c81-4f15-bb07-6e2770fc5fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:00.637334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1190/2353] 10.222.0.64 () {58 vars in 1018 bytes} [Mon Jun 27 09:39:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:00.686261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d4abf-422a-492e-b084-cf9487967a02 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:00.705301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d4abf-422a-492e-b084-cf9487967a02 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:00.713741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbfcdb-4589-46fd-9ba5-272161a60368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:00.714885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbfcdb-4589-46fd-9ba5-272161a60368 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:00.716356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbfcdb-4589-46fd-9ba5-272161a60368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:00.721127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbfcdb-4589-46fd-9ba5-272161a60368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:00.746611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbfcdb-4589-46fd-9ba5-272161a60368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:00.750637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbfcdb-4589-46fd-9ba5-272161a60368 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:00.780427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1164/2354] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:39:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:39:02.544773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15798ec9-4c81-4f15-bb07-6e2770fc5fd3 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:02.546992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-15798ec9-4c81-4f15-bb07-6e2770fc5fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:02.689894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:02.723191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15798ec9-4c81-4f15-bb07-6e2770fc5fd3 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:02.734917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:02.735900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:02.737438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:02.737726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] PATH_INFO: `/v3/users/bc51b5e3810d4bb382942272926293dd` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:02.740301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc51b5e3810d4bb382942272926293dd)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:02.746223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:03.056904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc51b5e3810d4bb382942272926293dd'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:03.496326 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc51b5e3810d4bb382942272926293dd'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:03.500158 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc51b5e3810d4bb382942272926293dd'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:03.515651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc51b5e3810d4bb382942272926293dd'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:03.517600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc51b5e3810d4bb382942272926293dd'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:03.614163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1191/2355] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:39:02 2022] DELETE /identity/v3/users/bc51b5e3810d4bb382942272926293dd => generated 0 bytes in 1070 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:03.618686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de010283-d36b-49ac-8b71-7c03cd43469a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:03.622717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de010283-d36b-49ac-8b71-7c03cd43469a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:03.622717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de010283-d36b-49ac-8b71-7c03cd43469a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:03.657359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-de010283-d36b-49ac-8b71-7c03cd43469a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:03.663510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-de010283-d36b-49ac-8b71-7c03cd43469a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:03.731438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:03.793287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1165/2356] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:03.826416 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:03.828166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:03.924680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5868c-a325-4f0a-a6bf-591d6c92de46 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:03.928674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b0820-0454-4974-acbd-8b6e42220c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:03.929398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b0820-0454-4974-acbd-8b6e42220c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:03.929680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b0820-0454-4974-acbd-8b6e42220c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:03.930038 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b0820-0454-4974-acbd-8b6e42220c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:03.943541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b0820-0454-4974-acbd-8b6e42220c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:03.945085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b0820-0454-4974-acbd-8b6e42220c67 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:03.958433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1192/2357] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:39:04.494369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de010283-d36b-49ac-8b71-7c03cd43469a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:04.495843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-de010283-d36b-49ac-8b71-7c03cd43469a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:04.638726 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de010283-d36b-49ac-8b71-7c03cd43469a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:04.645557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a032db-afb4-4cf3-9e0b-cf32b22c5540 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:04.647305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a032db-afb4-4cf3-9e0b-cf32b22c5540 service placement] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:04.647893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a032db-afb4-4cf3-9e0b-cf32b22c5540 service placement] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:04.648352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a032db-afb4-4cf3-9e0b-cf32b22c5540 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:04.650640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-91a032db-afb4-4cf3-9e0b-cf32b22c5540 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:04.664716 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a032db-afb4-4cf3-9e0b-cf32b22c5540 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:04.667178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a032db-afb4-4cf3-9e0b-cf32b22c5540 service placement] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:04.680564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1166/2358] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:39:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:39:05.663404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b0820-0454-4974-acbd-8b6e42220c67 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:05.665334 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-380b0820-0454-4974-acbd-8b6e42220c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:05.724713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b0820-0454-4974-acbd-8b6e42220c67 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:05.736361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:05.753980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:05.754479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:05.755820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] PATH_INFO: `/v3/projects/463d49496646486883fc78b78f67232c` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:05.757555 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=463d49496646486883fc78b78f67232c)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:05.762567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:05.985347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '463d49496646486883fc78b78f67232c', 'request_id': 'req-e8c42123-cce2-4b5d-a174-e8844194d935'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:06.089258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '463d49496646486883fc78b78f67232c', 'request_id': 'req-e8c42123-cce2-4b5d-a174-e8844194d935'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:06.098773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '463d49496646486883fc78b78f67232c', 'request_id': 'req-e8c42123-cce2-4b5d-a174-e8844194d935'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:06.101767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:06.129029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] The token cache is being invalidate because project 463d49496646486883fc78b78f67232c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:39:06.129322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:06.129980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e8c42123-cce2-4b5d-a174-e8844194d935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:06.131599 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1193/2359] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:39:05 2022] DELETE /identity/v3/projects/463d49496646486883fc78b78f67232c => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:06.137207 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7db6-53dd-4034-bdb7-a75d826b628a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:06.137744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7db6-53dd-4034-bdb7-a75d826b628a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:06.137985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee7db6-53dd-4034-bdb7-a75d826b628a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:06.181856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-6aee7db6-53dd-4034-bdb7-a75d826b628a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:06.183730 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-6aee7db6-53dd-4034-bdb7-a75d826b628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:06.299148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1167/2360] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:06.308145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d050c53-403e-438a-8615-d25b4b7513df None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:06.308145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d050c53-403e-438a-8615-d25b4b7513df None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:06.308684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d050c53-403e-438a-8615-d25b4b7513df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:06.333243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8d050c53-403e-438a-8615-d25b4b7513df None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:06.334816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8d050c53-403e-438a-8615-d25b4b7513df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:06.346657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1194/2361] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:06.352567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7db6-53dd-4034-bdb7-a75d826b628a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:06.365465 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee7db6-53dd-4034-bdb7-a75d826b628a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:06.378268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d975ff-9c3a-461f-a60f-0fd18708a481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:06.378944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d975ff-9c3a-461f-a60f-0fd18708a481 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:06.379226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d975ff-9c3a-461f-a60f-0fd18708a481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:06.380076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d975ff-9c3a-461f-a60f-0fd18708a481 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:06.380693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d975ff-9c3a-461f-a60f-0fd18708a481 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:39:06.380693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d975ff-9c3a-461f-a60f-0fd18708a481 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:06.382189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d975ff-9c3a-461f-a60f-0fd18708a481 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:06.398342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1168/2362] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:39:06.402967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d050c53-403e-438a-8615-d25b4b7513df None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:06.413890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d050c53-403e-438a-8615-d25b4b7513df None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:06.420293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf310bf-aa6f-40aa-8183-087574966f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:06.421375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf310bf-aa6f-40aa-8183-087574966f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:06.421905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf310bf-aa6f-40aa-8183-087574966f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:06.422515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf310bf-aa6f-40aa-8183-087574966f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:06.423441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf310bf-aa6f-40aa-8183-087574966f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:06.426587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf310bf-aa6f-40aa-8183-087574966f98 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:06.533186 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1195/2363] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:39:06 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:39:06.537385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d975ff-9c3a-461f-a60f-0fd18708a481 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:06.560606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d975ff-9c3a-461f-a60f-0fd18708a481 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:06.622938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:06.678415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb878a8-1172-4abc-953a-33f2ab7a7457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:06.679580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb878a8-1172-4abc-953a-33f2ab7a7457 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:06.680144 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb878a8-1172-4abc-953a-33f2ab7a7457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:06.680652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb878a8-1172-4abc-953a-33f2ab7a7457 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:06.681647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb878a8-1172-4abc-953a-33f2ab7a7457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:06.685700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb878a8-1172-4abc-953a-33f2ab7a7457 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:06.801810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1169/2364] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:39:06 2022] POST /identity/v3/users => generated 286 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:39:06.805850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf310bf-aa6f-40aa-8183-087574966f98 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:06.829999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf310bf-aa6f-40aa-8183-087574966f98 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:06.829999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fc9a-3fff-4a5d-89a1-4ce74b252bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:06.829999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fc9a-3fff-4a5d-89a1-4ce74b252bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:06.829999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fc9a-3fff-4a5d-89a1-4ce74b252bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:06.830671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047fc9a-3fff-4a5d-89a1-4ce74b252bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:06.830671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047fc9a-3fff-4a5d-89a1-4ce74b252bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:06.831444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047fc9a-3fff-4a5d-89a1-4ce74b252bf6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:06.847476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1196/2365] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:39:06.853281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb878a8-1172-4abc-953a-33f2ab7a7457 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:06.864931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb878a8-1172-4abc-953a-33f2ab7a7457 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:06.870392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c883b-07b1-4330-8fd9-e8511c5f50d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:06.871371 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c883b-07b1-4330-8fd9-e8511c5f50d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:06.871782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c883b-07b1-4330-8fd9-e8511c5f50d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:06.872132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c883b-07b1-4330-8fd9-e8511c5f50d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:06.872968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c883b-07b1-4330-8fd9-e8511c5f50d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:06.874674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c883b-07b1-4330-8fd9-e8511c5f50d2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:06.886615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1170/2366] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:39:06.891986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fc9a-3fff-4a5d-89a1-4ce74b252bf6 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:06.905209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047fc9a-3fff-4a5d-89a1-4ce74b252bf6 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:06.912588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090422cb-b7b2-4a53-9e5a-9447f3163e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:06.913769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090422cb-b7b2-4a53-9e5a-9447f3163e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:06.917066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090422cb-b7b2-4a53-9e5a-9447f3163e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:06.917066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090422cb-b7b2-4a53-9e5a-9447f3163e97 admin admin] PATH_INFO: `/v3/projects/9964ff08922043d7990b6389c21b4ee8/users/d5c0d7655f444171862b8555e0ded9cf/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:06.938542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090422cb-b7b2-4a53-9e5a-9447f3163e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9964ff08922043d7990b6389c21b4ee8, user_id=d5c0d7655f444171862b8555e0ded9cf, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:06.944838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090422cb-b7b2-4a53-9e5a-9447f3163e97 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:07.010819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1197/2367] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:39:06 2022] PUT /identity/v3/projects/9964ff08922043d7990b6389c21b4ee8/users/d5c0d7655f444171862b8555e0ded9cf/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:07.025297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ff95a-2ee5-427d-b2f1-9512cb5ef9e9 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:07.025297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ff95a-2ee5-427d-b2f1-9512cb5ef9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:07.025297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ff95a-2ee5-427d-b2f1-9512cb5ef9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:07.055606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1c2ff95a-2ee5-427d-b2f1-9512cb5ef9e9 None None] MFA Rules not processed for user `d5c0d7655f444171862b8555e0ded9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:07.056849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1c2ff95a-2ee5-427d-b2f1-9512cb5ef9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:07.141322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1171/2368] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:07.153785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090422cb-b7b2-4a53-9e5a-9447f3163e97 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:07.155728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-090422cb-b7b2-4a53-9e5a-9447f3163e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:07.248389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090422cb-b7b2-4a53-9e5a-9447f3163e97 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:07.252277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda577f9-7124-42f3-a9ba-334a48fac579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:07.253026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda577f9-7124-42f3-a9ba-334a48fac579 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:07.253315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda577f9-7124-42f3-a9ba-334a48fac579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:07.253591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda577f9-7124-42f3-a9ba-334a48fac579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:07.266222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda577f9-7124-42f3-a9ba-334a48fac579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:07.267809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda577f9-7124-42f3-a9ba-334a48fac579 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:07.292982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1198/2369] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:39:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:39:07.320995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ff95a-2ee5-427d-b2f1-9512cb5ef9e9 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:07.399759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ff95a-2ee5-427d-b2f1-9512cb5ef9e9 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:07.405011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:07.405962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:07.406252 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:07.406533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] PATH_INFO: `/v3/users/d5c0d7655f444171862b8555e0ded9cf` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:07.408412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c0d7655f444171862b8555e0ded9cf)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:07.410199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:07.681238 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:07.685640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c0d7655f444171862b8555e0ded9cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:07.687548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c0d7655f444171862b8555e0ded9cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:07.708302 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c0d7655f444171862b8555e0ded9cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:07.709073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c0d7655f444171862b8555e0ded9cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:07.741654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c0d7655f444171862b8555e0ded9cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:07.840478 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1172/2370] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:39:07 2022] DELETE /identity/v3/users/d5c0d7655f444171862b8555e0ded9cf => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:07.845310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580548f-8067-4315-a42d-2e0e247d02b8 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:07.845801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580548f-8067-4315-a42d-2e0e247d02b8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:07.845906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580548f-8067-4315-a42d-2e0e247d02b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:07.877660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3580548f-8067-4315-a42d-2e0e247d02b8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:07.879974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3580548f-8067-4315-a42d-2e0e247d02b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:08.020772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1199/2371] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:39:08.046514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:08.047974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:08.159753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f900806-e7c0-4977-b7b9-388238950cf4 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:08.164002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4347d33-35c4-400f-a770-0766010f0185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:08.164751 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4347d33-35c4-400f-a770-0766010f0185 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:08.165107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4347d33-35c4-400f-a770-0766010f0185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:08.165709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4347d33-35c4-400f-a770-0766010f0185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:08.188671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4347d33-35c4-400f-a770-0766010f0185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:08.190323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4347d33-35c4-400f-a770-0766010f0185 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:08.205982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1173/2372] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:39:08.840032 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580548f-8067-4315-a42d-2e0e247d02b8 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:08.841806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3580548f-8067-4315-a42d-2e0e247d02b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:08.869431 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580548f-8067-4315-a42d-2e0e247d02b8 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:08.879105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:08.883593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:08.883912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:08.884466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] PATH_INFO: `/v3/projects/9964ff08922043d7990b6389c21b4ee8` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:08.891189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9964ff08922043d7990b6389c21b4ee8)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:08.893885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:08.913819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:09.055291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9964ff08922043d7990b6389c21b4ee8', 'request_id': 'req-16d5cd96-4eac-42bd-ac39-5f72c4e21221'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:09.137359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9964ff08922043d7990b6389c21b4ee8', 'request_id': 'req-16d5cd96-4eac-42bd-ac39-5f72c4e21221'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:09.145058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9964ff08922043d7990b6389c21b4ee8', 'request_id': 'req-16d5cd96-4eac-42bd-ac39-5f72c4e21221'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:09.168653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] The token cache is being invalidate because project 9964ff08922043d7990b6389c21b4ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:39:09.169328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:09.171168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16d5cd96-4eac-42bd-ac39-5f72c4e21221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:09.174295 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1200/2373] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:39:08 2022] DELETE /identity/v3/projects/9964ff08922043d7990b6389c21b4ee8 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:09.179453 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63334f1b-8395-41a8-938b-0d7a5f6be380 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:09.180599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63334f1b-8395-41a8-938b-0d7a5f6be380 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:09.181434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63334f1b-8395-41a8-938b-0d7a5f6be380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:09.213198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-63334f1b-8395-41a8-938b-0d7a5f6be380 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:09.214800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-63334f1b-8395-41a8-938b-0d7a5f6be380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:09.364100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1174/2374] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:39:09.372000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accb628-11ea-464a-9f50-8d496dd754db None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:09.372293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accb628-11ea-464a-9f50-8d496dd754db None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:09.372572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accb628-11ea-464a-9f50-8d496dd754db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:09.417793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5accb628-11ea-464a-9f50-8d496dd754db None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:09.419938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5accb628-11ea-464a-9f50-8d496dd754db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:09.449403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63334f1b-8395-41a8-938b-0d7a5f6be380 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:09.449545 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1201/2375] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:09.496015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63334f1b-8395-41a8-938b-0d7a5f6be380 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:09.525626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68009b46-b211-4639-8eb9-a0c785b6a9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:09.528178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68009b46-b211-4639-8eb9-a0c785b6a9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:09.528178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68009b46-b211-4639-8eb9-a0c785b6a9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:09.528178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68009b46-b211-4639-8eb9-a0c785b6a9b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:09.534049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68009b46-b211-4639-8eb9-a0c785b6a9b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:39:09.535226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68009b46-b211-4639-8eb9-a0c785b6a9b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:09.536936 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68009b46-b211-4639-8eb9-a0c785b6a9b1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:09.580258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1175/2376] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:39:09.584701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5accb628-11ea-464a-9f50-8d496dd754db None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:09.645653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5accb628-11ea-464a-9f50-8d496dd754db None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:09.645653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621b1ed-a6d6-4cb4-8168-1b75aef34855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:09.645653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621b1ed-a6d6-4cb4-8168-1b75aef34855 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:09.645653 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621b1ed-a6d6-4cb4-8168-1b75aef34855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:09.646227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621b1ed-a6d6-4cb4-8168-1b75aef34855 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:09.646227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621b1ed-a6d6-4cb4-8168-1b75aef34855 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:09.646227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621b1ed-a6d6-4cb4-8168-1b75aef34855 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:10.125563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1202/2377] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:39:09 2022] POST /identity/v3/projects => generated 377 bytes in 542 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:39:10.129902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68009b46-b211-4639-8eb9-a0c785b6a9b1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:10.142091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68009b46-b211-4639-8eb9-a0c785b6a9b1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:10.210240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:10.220560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf97fe5-99c5-4b7f-a5ee-b9cfbc5480dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:10.221242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf97fe5-99c5-4b7f-a5ee-b9cfbc5480dd admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:10.221528 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf97fe5-99c5-4b7f-a5ee-b9cfbc5480dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:10.221807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf97fe5-99c5-4b7f-a5ee-b9cfbc5480dd admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:10.222538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf97fe5-99c5-4b7f-a5ee-b9cfbc5480dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:10.224518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf97fe5-99c5-4b7f-a5ee-b9cfbc5480dd admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:10.367391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1176/2378] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:39:10 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:39:10.372379 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621b1ed-a6d6-4cb4-8168-1b75aef34855 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:10.391252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621b1ed-a6d6-4cb4-8168-1b75aef34855 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:10.393123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccab723-180f-4b04-b808-32ccfae53078 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:10.393501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccab723-180f-4b04-b808-32ccfae53078 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:10.393799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccab723-180f-4b04-b808-32ccfae53078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:10.400441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a51e6-7ae5-48bf-bebc-e77bf5b29d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:10.401269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a51e6-7ae5-48bf-bebc-e77bf5b29d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:10.401655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a51e6-7ae5-48bf-bebc-e77bf5b29d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:10.402030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a51e6-7ae5-48bf-bebc-e77bf5b29d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:10.402955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a51e6-7ae5-48bf-bebc-e77bf5b29d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:10.406177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a51e6-7ae5-48bf-bebc-e77bf5b29d38 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:10.425656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1203/2379] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:39:10.426871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a51e6-7ae5-48bf-bebc-e77bf5b29d38 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:10.448737 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a51e6-7ae5-48bf-bebc-e77bf5b29d38 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:10.461474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0e173-03d3-4984-9835-a0b1e5c4ab02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:10.462799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0e173-03d3-4984-9835-a0b1e5c4ab02 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:10.463184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0e173-03d3-4984-9835-a0b1e5c4ab02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:10.463670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0e173-03d3-4984-9835-a0b1e5c4ab02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:10.464588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0e173-03d3-4984-9835-a0b1e5c4ab02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:10.466549 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0e173-03d3-4984-9835-a0b1e5c4ab02 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:10.486615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1204/2380] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:39:10.491160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0e173-03d3-4984-9835-a0b1e5c4ab02 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:10.503076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0e173-03d3-4984-9835-a0b1e5c4ab02 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:10.520888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9561c04-8ee9-4740-8ddd-15b001db8ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:10.522656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9561c04-8ee9-4740-8ddd-15b001db8ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:10.523042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9561c04-8ee9-4740-8ddd-15b001db8ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:10.523960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9561c04-8ee9-4740-8ddd-15b001db8ce7 admin admin] PATH_INFO: `/v3/projects/1d5cbd080e204408aa51c29ec2cf1422/users/2720d45854ab4a61b3b22dd43a523d4b/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:10.550684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9561c04-8ee9-4740-8ddd-15b001db8ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d5cbd080e204408aa51c29ec2cf1422, user_id=2720d45854ab4a61b3b22dd43a523d4b, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:10.553742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9561c04-8ee9-4740-8ddd-15b001db8ce7 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:10.591769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9ccab723-180f-4b04-b808-32ccfae53078 None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:10.596524 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9ccab723-180f-4b04-b808-32ccfae53078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:10.644265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1205/2381] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:39:10 2022] PUT /identity/v3/projects/1d5cbd080e204408aa51c29ec2cf1422/users/2720d45854ab4a61b3b22dd43a523d4b/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:10.650071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f83f1-3f9f-4109-96dc-72c81ed47e33 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:10.651605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f83f1-3f9f-4109-96dc-72c81ed47e33 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:10.651605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f83f1-3f9f-4109-96dc-72c81ed47e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:10.691716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-bc6f83f1-3f9f-4109-96dc-72c81ed47e33 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:10.696811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-bc6f83f1-3f9f-4109-96dc-72c81ed47e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:10.760819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1177/2382] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:39:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:39:10.774969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccab723-180f-4b04-b808-32ccfae53078 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:10.777220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9ccab723-180f-4b04-b808-32ccfae53078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:10.814995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1206/2383] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:10.826211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f83f1-3f9f-4109-96dc-72c81ed47e33 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:10.828033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-bc6f83f1-3f9f-4109-96dc-72c81ed47e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:10.872458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:10.935216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f83f1-3f9f-4109-96dc-72c81ed47e33 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:10.952476 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f6301-9af1-45c2-a50d-4c52a0e55f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:10.953149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f6301-9af1-45c2-a50d-4c52a0e55f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:10.953432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f6301-9af1-45c2-a50d-4c52a0e55f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:10.953711 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f6301-9af1-45c2-a50d-4c52a0e55f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:10.968012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f6301-9af1-45c2-a50d-4c52a0e55f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:10.969723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f6301-9af1-45c2-a50d-4c52a0e55f03 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:10.983291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1207/2384] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:39:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:39:11.007304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccab723-180f-4b04-b808-32ccfae53078 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:11.071718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:11.104083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07846e-e005-436b-97b5-cabec1ba04ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:11.104701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07846e-e005-436b-97b5-cabec1ba04ef service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:11.105043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07846e-e005-436b-97b5-cabec1ba04ef service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:11.105331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07846e-e005-436b-97b5-cabec1ba04ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:11.194122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07846e-e005-436b-97b5-cabec1ba04ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:11.197048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07846e-e005-436b-97b5-cabec1ba04ef service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:11.209669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1178/2385] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:39:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:39:16.672201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8fda-799e-4c46-bf2c-606aaa2c6b36 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:16.672795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8fda-799e-4c46-bf2c-606aaa2c6b36 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:16.672795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8fda-799e-4c46-bf2c-606aaa2c6b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:16.704531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-441c8fda-799e-4c46-bf2c-606aaa2c6b36 None None] MFA Rules not processed for user `5778103c646049f5944934e775cba9da`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:16.711994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-441c8fda-799e-4c46-bf2c-606aaa2c6b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:16.815214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1208/2386] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:21.205163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=37 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:22.834378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d780c26-b810-43fd-8a26-027596eb36c7 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:22.834969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d780c26-b810-43fd-8a26-027596eb36c7 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:22.835072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d780c26-b810-43fd-8a26-027596eb36c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:22.862740 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9d780c26-b810-43fd-8a26-027596eb36c7 None None] MFA Rules not processed for user `2720d45854ab4a61b3b22dd43a523d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:22.864005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9d780c26-b810-43fd-8a26-027596eb36c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:22.930480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1179/2387] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:22.934953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c8fda-799e-4c46-bf2c-606aaa2c6b36 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:22.946927 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c8fda-799e-4c46-bf2c-606aaa2c6b36 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:22.952214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07583f02-4b8e-4849-ba26-f5dcbd75094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:22.952887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07583f02-4b8e-4849-ba26-f5dcbd75094d admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:22.953186 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07583f02-4b8e-4849-ba26-f5dcbd75094d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:22.953470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07583f02-4b8e-4849-ba26-f5dcbd75094d admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:22.954180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07583f02-4b8e-4849-ba26-f5dcbd75094d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:22.956207 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07583f02-4b8e-4849-ba26-f5dcbd75094d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:23.100189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1209/2388] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:39:22 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:39:23.109876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d780c26-b810-43fd-8a26-027596eb36c7 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:23.124479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d780c26-b810-43fd-8a26-027596eb36c7 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:23.240006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2115bd-7f66-412c-b08d-2dca26fe8b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:23.240725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2115bd-7f66-412c-b08d-2dca26fe8b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:23.244253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2115bd-7f66-412c-b08d-2dca26fe8b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:23.245381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2115bd-7f66-412c-b08d-2dca26fe8b5e admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:23.245381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2115bd-7f66-412c-b08d-2dca26fe8b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:23.247798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2115bd-7f66-412c-b08d-2dca26fe8b5e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:23.336838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1180/2389] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:39:23 2022] POST /identity/v3/users => generated 296 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:39:23.340701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07583f02-4b8e-4849-ba26-f5dcbd75094d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:23.358474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07583f02-4b8e-4849-ba26-f5dcbd75094d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:23.370933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480defef-6590-4d5f-8557-5e0228c02631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:23.372009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480defef-6590-4d5f-8557-5e0228c02631 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:23.372451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480defef-6590-4d5f-8557-5e0228c02631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:23.372907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480defef-6590-4d5f-8557-5e0228c02631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:23.373815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480defef-6590-4d5f-8557-5e0228c02631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:23.375654 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480defef-6590-4d5f-8557-5e0228c02631 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:23.401987 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1210/2390] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:39:23.410276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2115bd-7f66-412c-b08d-2dca26fe8b5e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:23.426220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2115bd-7f66-412c-b08d-2dca26fe8b5e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:23.432692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa763831-5180-47a5-ae3c-b1ebccaa8058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:23.434109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa763831-5180-47a5-ae3c-b1ebccaa8058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:23.434616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa763831-5180-47a5-ae3c-b1ebccaa8058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:23.434996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa763831-5180-47a5-ae3c-b1ebccaa8058 admin admin] PATH_INFO: `/v3/projects/cac420d7f4d849e7b14b7f0c37dbf3bc/users/4987f547ce064257a0b180c83ef14bef/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:23.491211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa763831-5180-47a5-ae3c-b1ebccaa8058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac420d7f4d849e7b14b7f0c37dbf3bc, user_id=4987f547ce064257a0b180c83ef14bef, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:23.494926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa763831-5180-47a5-ae3c-b1ebccaa8058 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:23.614819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1181/2391] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:39:23 2022] PUT /identity/v3/projects/cac420d7f4d849e7b14b7f0c37dbf3bc/users/4987f547ce064257a0b180c83ef14bef/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:23.618913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480defef-6590-4d5f-8557-5e0228c02631 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:23.635695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480defef-6590-4d5f-8557-5e0228c02631 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:23.811970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabe194-517c-41f7-86a2-e25fe819e3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:23.812786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabe194-517c-41f7-86a2-e25fe819e3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:23.813075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabe194-517c-41f7-86a2-e25fe819e3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:23.813352 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabe194-517c-41f7-86a2-e25fe819e3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:23.814187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabe194-517c-41f7-86a2-e25fe819e3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:23.823435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabe194-517c-41f7-86a2-e25fe819e3c4 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:23.847293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1211/2392] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:39:23.852401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa763831-5180-47a5-ae3c-b1ebccaa8058 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:23.870844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa763831-5180-47a5-ae3c-b1ebccaa8058 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:23.884735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e5187-531b-49e6-b22e-3425eaf22075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:23.885913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e5187-531b-49e6-b22e-3425eaf22075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:23.886235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e5187-531b-49e6-b22e-3425eaf22075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:23.886566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e5187-531b-49e6-b22e-3425eaf22075 admin admin] PATH_INFO: `/v3/projects/cac420d7f4d849e7b14b7f0c37dbf3bc/users/4987f547ce064257a0b180c83ef14bef/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:23.888708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e5187-531b-49e6-b22e-3425eaf22075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cac420d7f4d849e7b14b7f0c37dbf3bc, user_id=4987f547ce064257a0b180c83ef14bef, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:23.891421 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e5187-531b-49e6-b22e-3425eaf22075 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:23.934882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1182/2393] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:39:23 2022] PUT /identity/v3/projects/cac420d7f4d849e7b14b7f0c37dbf3bc/users/4987f547ce064257a0b180c83ef14bef/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:25.019650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7d1cc-91a1-475a-ba55-8baafc4cbe10 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:25.020179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7d1cc-91a1-475a-ba55-8baafc4cbe10 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:25.020528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7d1cc-91a1-475a-ba55-8baafc4cbe10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:25.042130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-84e7d1cc-91a1-475a-ba55-8baafc4cbe10 None None] MFA Rules not processed for user `cc8a087e97424d2890952be33695378f`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:25.043709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-84e7d1cc-91a1-475a-ba55-8baafc4cbe10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:25.157049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:25.190786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1212/2394] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:27.421193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c6f5a-7bf6-40ed-9061-df8a404e4f4c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:27.423316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c6f5a-7bf6-40ed-9061-df8a404e4f4c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:27.423316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c6f5a-7bf6-40ed-9061-df8a404e4f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:27.450088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-585c6f5a-7bf6-40ed-9061-df8a404e4f4c None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:27.453464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-585c6f5a-7bf6-40ed-9061-df8a404e4f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:27.568665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:27.675023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1183/2395] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:39:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:27.693866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7d1cc-91a1-475a-ba55-8baafc4cbe10 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:27.813369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7d1cc-91a1-475a-ba55-8baafc4cbe10 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:27.819495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12823ab2-a6fa-440f-8ed2-28e4e43cbd06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:27.820333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12823ab2-a6fa-440f-8ed2-28e4e43cbd06 service swift] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:27.820711 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12823ab2-a6fa-440f-8ed2-28e4e43cbd06 service swift] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:27.821079 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12823ab2-a6fa-440f-8ed2-28e4e43cbd06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:27.832839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12823ab2-a6fa-440f-8ed2-28e4e43cbd06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:27.834799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12823ab2-a6fa-440f-8ed2-28e4e43cbd06 service swift] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:27.849357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1213/2396] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:39:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:39:28.527513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e08c74-59dc-4907-b9c5-69bd99f58379 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:28.527965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e08c74-59dc-4907-b9c5-69bd99f58379 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:28.528114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e08c74-59dc-4907-b9c5-69bd99f58379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:28.560410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-16e08c74-59dc-4907-b9c5-69bd99f58379 None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:28.561701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-16e08c74-59dc-4907-b9c5-69bd99f58379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:28.581651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1184/2397] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:39:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:28.594052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12823ab2-a6fa-440f-8ed2-28e4e43cbd06 service swift] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:28.606401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12823ab2-a6fa-440f-8ed2-28e4e43cbd06 service swift] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:28.614201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd37a6b-06c6-4d6f-82a8-304fa2cf06ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:28.614906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd37a6b-06c6-4d6f-82a8-304fa2cf06ca service swift] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:28.615196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd37a6b-06c6-4d6f-82a8-304fa2cf06ca service swift] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:28.615467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd37a6b-06c6-4d6f-82a8-304fa2cf06ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:28.635698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd37a6b-06c6-4d6f-82a8-304fa2cf06ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:28.646074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd37a6b-06c6-4d6f-82a8-304fa2cf06ca service swift] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:28.666879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1214/2398] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:39:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:39:29.366995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d22e7-58a6-4863-85c5-4a97735c098c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:29.368339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d22e7-58a6-4863-85c5-4a97735c098c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:29.368852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d22e7-58a6-4863-85c5-4a97735c098c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:29.416107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-4f8d22e7-58a6-4863-85c5-4a97735c098c None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:29.418156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4f8d22e7-58a6-4863-85c5-4a97735c098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:29.434213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1185/2399] 10.222.0.64 () {58 vars in 967 bytes} [Mon Jun 27 09:39:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:39:29.468441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd37a6b-06c6-4d6f-82a8-304fa2cf06ca service swift] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:29.490268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd37a6b-06c6-4d6f-82a8-304fa2cf06ca service swift] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:29.506619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b92ddd-5b25-49a5-84e8-57c84311917c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:29.509030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b92ddd-5b25-49a5-84e8-57c84311917c service swift] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:29.509371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b92ddd-5b25-49a5-84e8-57c84311917c service swift] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:29.509669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b92ddd-5b25-49a5-84e8-57c84311917c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:29.564179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b92ddd-5b25-49a5-84e8-57c84311917c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:29.564179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b92ddd-5b25-49a5-84e8-57c84311917c service swift] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:29.593548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1215/2400] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:39:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:39:30.539805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc7356-4d6c-43f6-9157-946a38d58e15 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:30.539805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc7356-4d6c-43f6-9157-946a38d58e15 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:30.539805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc7356-4d6c-43f6-9157-946a38d58e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:30.567557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-64dc7356-4d6c-43f6-9157-946a38d58e15 None None] MFA Rules not processed for user `09f242a8f51545199a159f6992ee74f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:30.568887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-64dc7356-4d6c-43f6-9157-946a38d58e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:30.685375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1186/2401] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:31.189016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b816296-7079-473c-899d-af3ba734e9ed None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:31.189016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b816296-7079-473c-899d-af3ba734e9ed None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:31.189016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b816296-7079-473c-899d-af3ba734e9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:31.228834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-7b816296-7079-473c-899d-af3ba734e9ed None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:31.235259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7b816296-7079-473c-899d-af3ba734e9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:31.262690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1216/2402] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:39:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:39:31.391389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc7356-4d6c-43f6-9157-946a38d58e15 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:31.391389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc7356-4d6c-43f6-9157-946a38d58e15 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:31.391389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7b47c-163c-4b36-89f4-135139b17dde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:31.391389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7b47c-163c-4b36-89f4-135139b17dde service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:31.395193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7b47c-163c-4b36-89f4-135139b17dde service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:31.395193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7b47c-163c-4b36-89f4-135139b17dde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:31.395193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7b47c-163c-4b36-89f4-135139b17dde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:31.395193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7b47c-163c-4b36-89f4-135139b17dde service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:31.402324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1187/2403] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:39:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:39:32.500515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc7d3-4357-4c1e-8a7a-1e7f26458c31 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:32.501100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc7d3-4357-4c1e-8a7a-1e7f26458c31 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:32.501581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc7d3-4357-4c1e-8a7a-1e7f26458c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:32.544763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-5b0dc7d3-4357-4c1e-8a7a-1e7f26458c31 None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:32.546177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5b0dc7d3-4357-4c1e-8a7a-1e7f26458c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:32.554362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1217/2404] 10.222.0.64 () {58 vars in 967 bytes} [Mon Jun 27 09:39:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:32.585491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7b47c-163c-4b36-89f4-135139b17dde service swift] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:32.602266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7b47c-163c-4b36-89f4-135139b17dde service swift] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:32.623045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd660-cadd-4f89-9011-cacaf8809bc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:32.623045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd660-cadd-4f89-9011-cacaf8809bc2 service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:32.623632 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd660-cadd-4f89-9011-cacaf8809bc2 service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:32.624032 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bd660-cadd-4f89-9011-cacaf8809bc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:32.642094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bd660-cadd-4f89-9011-cacaf8809bc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:32.644256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bd660-cadd-4f89-9011-cacaf8809bc2 service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:32.667065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1188/2405] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:39:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:39:35.705468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573bc7c-de88-431c-994b-8642664ef579 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:35.706006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573bc7c-de88-431c-994b-8642664ef579 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:35.706107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573bc7c-de88-431c-994b-8642664ef579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:35.801058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-0573bc7c-de88-431c-994b-8642664ef579 None None] MFA Rules not processed for user `4987f547ce064257a0b180c83ef14bef`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:35.803881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-0573bc7c-de88-431c-994b-8642664ef579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:35.977438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1218/2406] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:35.983152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd660-cadd-4f89-9011-cacaf8809bc2 service swift] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:35.984768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-382bd660-cadd-4f89-9011-cacaf8809bc2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:36.184753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bd660-cadd-4f89-9011-cacaf8809bc2 service swift] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:36.198591 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da665-b798-45e3-b005-0ef1fdf9aa69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:36.200162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da665-b798-45e3-b005-0ef1fdf9aa69 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:36.200162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da665-b798-45e3-b005-0ef1fdf9aa69 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:36.206638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da665-b798-45e3-b005-0ef1fdf9aa69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:36.317434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:36.329979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252da665-b798-45e3-b005-0ef1fdf9aa69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:36.332076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252da665-b798-45e3-b005-0ef1fdf9aa69 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:36.371117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1189/2407] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:39:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:39:36.666413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573bc7c-de88-431c-994b-8642664ef579 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:36.672989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-0573bc7c-de88-431c-994b-8642664ef579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:36.750002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:36.880075 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573bc7c-de88-431c-994b-8642664ef579 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:36.903469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:36.903469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:36.903469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:36.903826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] PATH_INFO: `/v3/users/fdf2758dad91482ea5da6bfdb5896313` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:36.907823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdf2758dad91482ea5da6bfdb5896313)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:36.914312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:37.201941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdf2758dad91482ea5da6bfdb5896313'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:37.277891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdf2758dad91482ea5da6bfdb5896313'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:37.279876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdf2758dad91482ea5da6bfdb5896313'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:37.306161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdf2758dad91482ea5da6bfdb5896313'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:37.307560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdf2758dad91482ea5da6bfdb5896313'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:37.354954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1219/2408] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:39:36 2022] DELETE /identity/v3/users/fdf2758dad91482ea5da6bfdb5896313 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:37.356123 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1553c-ff68-4d5c-8199-28fb1d2f9283 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:37.359120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1553c-ff68-4d5c-8199-28fb1d2f9283 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:37.359750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1553c-ff68-4d5c-8199-28fb1d2f9283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:37.429127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-e5d1553c-ff68-4d5c-8199-28fb1d2f9283 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:37.430547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e5d1553c-ff68-4d5c-8199-28fb1d2f9283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:37.549997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1190/2409] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:37.594973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:37.596982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:37.772676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a530bf6-d8c8-4b38-b41c-e422dc604c9c admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:37.783109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06a456-ef03-40eb-a2a6-93a62c6cbcbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:37.784189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06a456-ef03-40eb-a2a6-93a62c6cbcbe service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:37.784744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06a456-ef03-40eb-a2a6-93a62c6cbcbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:37.785258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06a456-ef03-40eb-a2a6-93a62c6cbcbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:37.798185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06a456-ef03-40eb-a2a6-93a62c6cbcbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:37.801475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06a456-ef03-40eb-a2a6-93a62c6cbcbe service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:37.823126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1220/2410] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:39:38.886477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1553c-ff68-4d5c-8199-28fb1d2f9283 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:38.888705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e5d1553c-ff68-4d5c-8199-28fb1d2f9283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:38.902177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1553c-ff68-4d5c-8199-28fb1d2f9283 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:38.907658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:38.909020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:38.909570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:38.910107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] PATH_INFO: `/v3/projects/0eb2a1214a1a4ae5a5211c18c2c39c1a` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:38.911720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eb2a1214a1a4ae5a5211c18c2c39c1a)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:38.914722 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:39.087785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eb2a1214a1a4ae5a5211c18c2c39c1a', 'request_id': 'req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:39.095321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eb2a1214a1a4ae5a5211c18c2c39c1a', 'request_id': 'req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:39.103931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eb2a1214a1a4ae5a5211c18c2c39c1a', 'request_id': 'req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:39.201267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] The token cache is being invalidate because project 0eb2a1214a1a4ae5a5211c18c2c39c1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:39:39.201267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:39.201267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:39:39.201267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1191/2411] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:39:38 2022] DELETE /identity/v3/projects/0eb2a1214a1a4ae5a5211c18c2c39c1a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:39:44.126831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6b06e-eec4-44ce-ba45-bbd0e7f3ee53 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:44.130086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6b06e-eec4-44ce-ba45-bbd0e7f3ee53 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:44.130086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6b06e-eec4-44ce-ba45-bbd0e7f3ee53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:44.148503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c9d6b06e-eec4-44ce-ba45-bbd0e7f3ee53 None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:44.150101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c9d6b06e-eec4-44ce-ba45-bbd0e7f3ee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:44.187768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:44.211893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1221/2412] 10.222.0.64 () {58 vars in 967 bytes} [Mon Jun 27 09:39:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:39:44.236777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:39:44.243119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:44.313164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f1f90-3d34-4341-962c-7edfd7ff3d6b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:39:44.318594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099aed8-fee8-4ad4-a2c2-ad9503766e83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:39:44.319357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099aed8-fee8-4ad4-a2c2-ad9503766e83 service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:44.319690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099aed8-fee8-4ad4-a2c2-ad9503766e83 service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:44.319992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099aed8-fee8-4ad4-a2c2-ad9503766e83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:44.328972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099aed8-fee8-4ad4-a2c2-ad9503766e83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:39:44.330494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099aed8-fee8-4ad4-a2c2-ad9503766e83 service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:39:44.339251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1192/2413] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:39:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:39:54.259313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=8 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:54.335323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:39:58.619566 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec456-d728-42a4-9663-ca721d23ab63 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:39:58.620079 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec456-d728-42a4-9663-ca721d23ab63 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:39:58.620172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec456-d728-42a4-9663-ca721d23ab63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:39:58.638368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ac9ec456-d728-42a4-9663-ca721d23ab63 None None] MFA Rules not processed for user `2720d45854ab4a61b3b22dd43a523d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:39:58.639609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ac9ec456-d728-42a4-9663-ca721d23ab63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:39:58.696432 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1222/2414] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:39:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:40:04.520010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099aed8-fee8-4ad4-a2c2-ad9503766e83 service swift] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:40:04.521543 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2099aed8-fee8-4ad4-a2c2-ad9503766e83 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:40:04.577173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099aed8-fee8-4ad4-a2c2-ad9503766e83 service swift] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:40:04.580777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36403437-4410-49db-995a-5087ea9b31c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:40:04.581416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36403437-4410-49db-995a-5087ea9b31c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:04.581690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36403437-4410-49db-995a-5087ea9b31c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:04.581960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36403437-4410-49db-995a-5087ea9b31c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:04.583756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-36403437-4410-49db-995a-5087ea9b31c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:40:04.646638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36403437-4410-49db-995a-5087ea9b31c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:40:04.648138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36403437-4410-49db-995a-5087ea9b31c6 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:40:04.658017 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1193/2415] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:40:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:40:08.374483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf1f15-fc2b-4956-b020-29b5faabcf94 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:08.374891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf1f15-fc2b-4956-b020-29b5faabcf94 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:08.375154 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf1f15-fc2b-4956-b020-29b5faabcf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:08.396273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-69bf1f15-fc2b-4956-b020-29b5faabcf94 None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:40:08.398196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-69bf1f15-fc2b-4956-b020-29b5faabcf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:40:08.405860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1223/2416] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:40:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:40:08.419146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36403437-4410-49db-995a-5087ea9b31c6 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:40:08.431082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36403437-4410-49db-995a-5087ea9b31c6 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:40:08.436297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e435c-adbc-4de6-af28-27cfd6807ed6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:40:08.436942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e435c-adbc-4de6-af28-27cfd6807ed6 service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:08.437353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e435c-adbc-4de6-af28-27cfd6807ed6 service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:08.437482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e435c-adbc-4de6-af28-27cfd6807ed6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:08.446511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e435c-adbc-4de6-af28-27cfd6807ed6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:40:08.448981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e435c-adbc-4de6-af28-27cfd6807ed6 service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:40:08.457641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1194/2417] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:40:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:40:18.389450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=28 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:40:18.454276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=56 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:40:25.978324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a2cac-ac8b-46e4-8be8-8eedae365c8c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:25.978787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a2cac-ac8b-46e4-8be8-8eedae365c8c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:25.978880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a2cac-ac8b-46e4-8be8-8eedae365c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:25.996733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8a0a2cac-ac8b-46e4-8be8-8eedae365c8c None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:40:25.997957 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8a0a2cac-ac8b-46e4-8be8-8eedae365c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:40:26.005040 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1224/2418] 10.222.0.64 () {58 vars in 967 bytes} [Mon Jun 27 09:40:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:40:26.016940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e435c-adbc-4de6-af28-27cfd6807ed6 service swift] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:40:26.027907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e435c-adbc-4de6-af28-27cfd6807ed6 service swift] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:40:26.032912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d2148-aeb8-4659-864a-2768df849a64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:40:26.033546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d2148-aeb8-4659-864a-2768df849a64 service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:26.033831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d2148-aeb8-4659-864a-2768df849a64 service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:26.034110 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d2148-aeb8-4659-864a-2768df849a64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:26.043038 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d2148-aeb8-4659-864a-2768df849a64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:40:26.044703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d2148-aeb8-4659-864a-2768df849a64 service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:40:26.052799 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1195/2419] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:40:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:40:26.519239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a2cac-ac8b-46e4-8be8-8eedae365c8c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:40:26.520664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8a0a2cac-ac8b-46e4-8be8-8eedae365c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:40:26.532963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a2cac-ac8b-46e4-8be8-8eedae365c8c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:40:26.536647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb75ea9-c60f-4982-85f9-25a7b1e26167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:40:26.537303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb75ea9-c60f-4982-85f9-25a7b1e26167 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:26.537585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb75ea9-c60f-4982-85f9-25a7b1e26167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:26.537860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb75ea9-c60f-4982-85f9-25a7b1e26167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:26.539672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-cbb75ea9-c60f-4982-85f9-25a7b1e26167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:40:26.603279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb75ea9-c60f-4982-85f9-25a7b1e26167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:40:26.604645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb75ea9-c60f-4982-85f9-25a7b1e26167 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:40:26.614676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1225/2420] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:40:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:40:29.103288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d2148-aeb8-4659-864a-2768df849a64 service swift] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:40:29.105444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7a9d2148-aeb8-4659-864a-2768df849a64 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:40:29.197425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d2148-aeb8-4659-864a-2768df849a64 service swift] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:40:29.202995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:40:29.204109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:29.204552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:29.205049 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] PATH_INFO: `/v3/users/09f242a8f51545199a159f6992ee74f1` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:29.206482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f242a8f51545199a159f6992ee74f1)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:40:29.208672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:40:29.361166 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f242a8f51545199a159f6992ee74f1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.361824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f242a8f51545199a159f6992ee74f1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.368813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f242a8f51545199a159f6992ee74f1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.369477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f242a8f51545199a159f6992ee74f1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.384190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-0ce0ae75-6f0a-40ed-a462-089740f58f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f242a8f51545199a159f6992ee74f1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.436382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1196/2421] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:40:29 2022] DELETE /identity/v3/users/09f242a8f51545199a159f6992ee74f1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:40:29.678173 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb75ea9-c60f-4982-85f9-25a7b1e26167 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:40:29.689752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb75ea9-c60f-4982-85f9-25a7b1e26167 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:40:29.752039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:40:29.752899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:29.753179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:29.753499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] PATH_INFO: `/v3/projects/604921be35cc4c258933114a5535800c` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:29.754684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=604921be35cc4c258933114a5535800c)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:40:29.756625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:40:29.870778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '604921be35cc4c258933114a5535800c', 'request_id': 'req-52229d1d-2f04-4695-8739-f67209305217'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.937848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '604921be35cc4c258933114a5535800c', 'request_id': 'req-52229d1d-2f04-4695-8739-f67209305217'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.943450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '604921be35cc4c258933114a5535800c', 'request_id': 'req-52229d1d-2f04-4695-8739-f67209305217'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.955226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] The token cache is being invalidate because project 604921be35cc4c258933114a5535800c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:40:29.955620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.956336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-52229d1d-2f04-4695-8739-f67209305217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:40:29.959115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1226/2422] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:40:29 2022] DELETE /identity/v3/projects/604921be35cc4c258933114a5535800c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:40:33.167813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c137e-3ab3-460f-8a7c-f7227c5ae54c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:33.168305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c137e-3ab3-460f-8a7c-f7227c5ae54c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:33.168401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c137e-3ab3-460f-8a7c-f7227c5ae54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:33.186402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-304c137e-3ab3-460f-8a7c-f7227c5ae54c None None] MFA Rules not processed for user `41ce5cc7ff504d64aac73e48097a4b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:40:33.187644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-304c137e-3ab3-460f-8a7c-f7227c5ae54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:40:33.240931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1197/2423] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:40:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:40:39.952261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:40:43.234361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:40:46.368161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6969-4789-4952-9433-7cf48d321023 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:40:46.368650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6969-4789-4952-9433-7cf48d321023 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:40:46.368770 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6969-4789-4952-9433-7cf48d321023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:40:46.386946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c4dd6969-4789-4952-9433-7cf48d321023 None None] MFA Rules not processed for user `ecf085030e8e4a65a7140fea2b64a7ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:40:46.388197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c4dd6969-4789-4952-9433-7cf48d321023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:40:46.442101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1227/2424] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:40:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:40:56.435592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=25 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:41:02.583671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c137e-3ab3-460f-8a7c-f7227c5ae54c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:02.585149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-304c137e-3ab3-460f-8a7c-f7227c5ae54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:02.644251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c137e-3ab3-460f-8a7c-f7227c5ae54c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:02.647946 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51cbb8-1957-4985-af31-df07d3e5561d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:02.648618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51cbb8-1957-4985-af31-df07d3e5561d service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:02.648928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51cbb8-1957-4985-af31-df07d3e5561d service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:02.649205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51cbb8-1957-4985-af31-df07d3e5561d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:02.650984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ef51cbb8-1957-4985-af31-df07d3e5561d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:02.738750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51cbb8-1957-4985-af31-df07d3e5561d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:02.740615 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51cbb8-1957-4985-af31-df07d3e5561d service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:02.753258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1198/2425] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:41:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:41:05.154882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd6969-4789-4952-9433-7cf48d321023 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:05.154882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c4dd6969-4789-4952-9433-7cf48d321023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:05.165853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd6969-4789-4952-9433-7cf48d321023 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:05.169873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:05.170481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:05.170765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:05.171219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:05.173218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:05.187823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:05.189297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:05.201316 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1228/2426] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:41:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:41:08.077407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf7197-67a6-4780-bb1d-d1ab9c74e958 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:08.078931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf7197-67a6-4780-bb1d-d1ab9c74e958 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:08.078931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf7197-67a6-4780-bb1d-d1ab9c74e958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:08.095276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9bcf7197-67a6-4780-bb1d-d1ab9c74e958 None None] MFA Rules not processed for user `cc8a087e97424d2890952be33695378f`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:41:08.096516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9bcf7197-67a6-4780-bb1d-d1ab9c74e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:08.152000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1199/2427] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:41:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:41:15.195631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=6 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:41:18.145250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=79 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:41:19.177593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:19.179152 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:19.235946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452821-62b5-4619-ac3c-81936749fcc8 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:19.239730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2854d-eada-463f-acf8-dee1183b419f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:19.240371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2854d-eada-463f-acf8-dee1183b419f service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:19.240655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2854d-eada-463f-acf8-dee1183b419f service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:19.240933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2854d-eada-463f-acf8-dee1183b419f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:19.242752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d0e2854d-eada-463f-acf8-dee1183b419f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:19.253721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2854d-eada-463f-acf8-dee1183b419f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:19.255199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2854d-eada-463f-acf8-dee1183b419f service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:19.264319 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1229/2428] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:41:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:41:29.260497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=26 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:41:33.781774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf7197-67a6-4780-bb1d-d1ab9c74e958 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:33.783300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9bcf7197-67a6-4780-bb1d-d1ab9c74e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:33.800189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf7197-67a6-4780-bb1d-d1ab9c74e958 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:33.806096 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:33.811407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:33.811407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:33.811407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:33.811407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:33.822232 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:33.823968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:33.835027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1200/2429] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:41:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:41:41.671625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2854d-eada-463f-acf8-dee1183b419f service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:41.680961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2854d-eada-463f-acf8-dee1183b419f service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:41.684567 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8350d1-d357-4b81-844d-3237138066b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:41.685198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8350d1-d357-4b81-844d-3237138066b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:41.685483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8350d1-d357-4b81-844d-3237138066b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:41.685765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8350d1-d357-4b81-844d-3237138066b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:41.687506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7a8350d1-d357-4b81-844d-3237138066b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:41.749226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8350d1-d357-4b81-844d-3237138066b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:41.750663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8350d1-d357-4b81-844d-3237138066b1 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:41.760464 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1230/2430] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:41:43.830242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=6 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:41:45.442276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:45.443838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:45.458439 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2925da2-af3c-4d57-878c-39a56e1fa991 service nova] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:45.462978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:45.463884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:45.464167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:45.464443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] PATH_INFO: `/v3/users/41ce5cc7ff504d64aac73e48097a4b01` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:45.465639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ce5cc7ff504d64aac73e48097a4b01)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:45.467570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:45.620530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ce5cc7ff504d64aac73e48097a4b01'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:45.621209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ce5cc7ff504d64aac73e48097a4b01'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:45.628040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ce5cc7ff504d64aac73e48097a4b01'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:45.628753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ce5cc7ff504d64aac73e48097a4b01'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:45.641634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b91c66e1-a95d-42e0-a444-dc68f7b72df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ce5cc7ff504d64aac73e48097a4b01'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:45.717800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1201/2431] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:41:45 2022] DELETE /identity/v3/users/41ce5cc7ff504d64aac73e48097a4b01 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:41:45.962750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8350d1-d357-4b81-844d-3237138066b1 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:45.972970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8350d1-d357-4b81-844d-3237138066b1 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:46.030718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:46.031616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.031901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.032179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] PATH_INFO: `/v3/projects/6a50c1d09d3247728aa1ef9fb69463fc` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.033367 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a50c1d09d3247728aa1ef9fb69463fc)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:46.035313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:46.155277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a50c1d09d3247728aa1ef9fb69463fc', 'request_id': 'req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:46.234823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a50c1d09d3247728aa1ef9fb69463fc', 'request_id': 'req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:46.240846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a50c1d09d3247728aa1ef9fb69463fc', 'request_id': 'req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:46.252390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] The token cache is being invalidate because project 6a50c1d09d3247728aa1ef9fb69463fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:41:46.252672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:46.253293 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-27e7bfb5-8e6d-4fd2-a2f0-6aa7763bae95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:41:46.254808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1231/2432] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:41:45 2022] DELETE /identity/v3/projects/6a50c1d09d3247728aa1ef9fb69463fc => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:41:46.259983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661d1c8-7d3e-4770-bdb4-c89a66c14166 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.260280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661d1c8-7d3e-4770-bdb4-c89a66c14166 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.261172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661d1c8-7d3e-4770-bdb4-c89a66c14166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.279423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-0661d1c8-7d3e-4770-bdb4-c89a66c14166 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:41:46.280641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0661d1c8-7d3e-4770-bdb4-c89a66c14166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:46.340006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1202/2433] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:41:46.347561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511d8d4-307b-4b3d-aa7d-7db8baea45a8 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.347938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511d8d4-307b-4b3d-aa7d-7db8baea45a8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.348223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511d8d4-307b-4b3d-aa7d-7db8baea45a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.366744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2511d8d4-307b-4b3d-aa7d-7db8baea45a8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:41:46.368849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2511d8d4-307b-4b3d-aa7d-7db8baea45a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:46.378006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1232/2434] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:41:46.382114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661d1c8-7d3e-4770-bdb4-c89a66c14166 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:46.392226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661d1c8-7d3e-4770-bdb4-c89a66c14166 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:46.396709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5d642-79ef-4e2a-8368-f7ccb0c93d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:46.397348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5d642-79ef-4e2a-8368-f7ccb0c93d5f admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.397625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5d642-79ef-4e2a-8368-f7ccb0c93d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.397895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5d642-79ef-4e2a-8368-f7ccb0c93d5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.398606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5d642-79ef-4e2a-8368-f7ccb0c93d5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:41:46.398906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5d642-79ef-4e2a-8368-f7ccb0c93d5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:46.400513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5d642-79ef-4e2a-8368-f7ccb0c93d5f admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:46.411724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1203/2435] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:41:46.415758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511d8d4-307b-4b3d-aa7d-7db8baea45a8 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:46.425242 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511d8d4-307b-4b3d-aa7d-7db8baea45a8 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:46.429847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ef0aa-6bd2-4655-9a81-4decefbdbce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:46.430488 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ef0aa-6bd2-4655-9a81-4decefbdbce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.430772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ef0aa-6bd2-4655-9a81-4decefbdbce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.431054 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ef0aa-6bd2-4655-9a81-4decefbdbce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.431772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ef0aa-6bd2-4655-9a81-4decefbdbce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:46.434143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ef0aa-6bd2-4655-9a81-4decefbdbce9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:46.447657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:41:46.563752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1233/2436] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:41:46 2022] POST /identity/v3/projects => generated 357 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:41:46.577449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5d642-79ef-4e2a-8368-f7ccb0c93d5f admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:46.587344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5d642-79ef-4e2a-8368-f7ccb0c93d5f admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:46.641785 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54999a8d-82a0-471f-9cf2-d74479c8b6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:46.642448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54999a8d-82a0-471f-9cf2-d74479c8b6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.642731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54999a8d-82a0-471f-9cf2-d74479c8b6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.643008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54999a8d-82a0-471f-9cf2-d74479c8b6c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.643750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54999a8d-82a0-471f-9cf2-d74479c8b6c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:46.645661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54999a8d-82a0-471f-9cf2-d74479c8b6c7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:46.652176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:41:46.707452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1204/2437] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:41:46 2022] POST /identity/v3/users => generated 279 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:41:46.711563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ef0aa-6bd2-4655-9a81-4decefbdbce9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:46.722016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ef0aa-6bd2-4655-9a81-4decefbdbce9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:46.726758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732394b-ad2a-4f9f-853e-743a7fcddd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:46.729732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732394b-ad2a-4f9f-853e-743a7fcddd1b admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.729732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732394b-ad2a-4f9f-853e-743a7fcddd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.729732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732394b-ad2a-4f9f-853e-743a7fcddd1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.729732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732394b-ad2a-4f9f-853e-743a7fcddd1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:46.730865 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732394b-ad2a-4f9f-853e-743a7fcddd1b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:46.742249 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1234/2438] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:41:46.746557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54999a8d-82a0-471f-9cf2-d74479c8b6c7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:46.756244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54999a8d-82a0-471f-9cf2-d74479c8b6c7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:46.760956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906d466-3ac8-47ac-b21c-781048194d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:46.761603 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906d466-3ac8-47ac-b21c-781048194d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.761887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906d466-3ac8-47ac-b21c-781048194d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.762164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906d466-3ac8-47ac-b21c-781048194d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.762881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906d466-3ac8-47ac-b21c-781048194d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:46.764492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906d466-3ac8-47ac-b21c-781048194d20 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:46.775091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1205/2439] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:41:46.779001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732394b-ad2a-4f9f-853e-743a7fcddd1b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:46.789025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732394b-ad2a-4f9f-853e-743a7fcddd1b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:46.794506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9597f-61cb-4972-a1b2-bd2cec880079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:46.795368 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9597f-61cb-4972-a1b2-bd2cec880079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.795690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9597f-61cb-4972-a1b2-bd2cec880079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.795977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9597f-61cb-4972-a1b2-bd2cec880079 admin admin] PATH_INFO: `/v3/projects/e16d7489b72b41ce95a200ea6bf7ab2c/users/23094d07d73f425398caa2499bd816ef/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.810739 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9597f-61cb-4972-a1b2-bd2cec880079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e16d7489b72b41ce95a200ea6bf7ab2c, user_id=23094d07d73f425398caa2499bd816ef, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:46.813430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9597f-61cb-4972-a1b2-bd2cec880079 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:46.880559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1235/2440] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:41:46 2022] PUT /identity/v3/projects/e16d7489b72b41ce95a200ea6bf7ab2c/users/23094d07d73f425398caa2499bd816ef/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:41:46.890297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa3a37-9dc1-4927-ba1c-c6ce943c0d13 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:46.891372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa3a37-9dc1-4927-ba1c-c6ce943c0d13 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:46.891372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa3a37-9dc1-4927-ba1c-c6ce943c0d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:46.918095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f9fa3a37-9dc1-4927-ba1c-c6ce943c0d13 None None] MFA Rules not processed for user `23094d07d73f425398caa2499bd816ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:41:46.919316 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f9fa3a37-9dc1-4927-ba1c-c6ce943c0d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:46.981329 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1206/2441] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:41:46 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:41:46.985704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9597f-61cb-4972-a1b2-bd2cec880079 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:47.050721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9597f-61cb-4972-a1b2-bd2cec880079 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:47.055362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed5282-907c-4afc-9ebe-525ed80b7657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:47.056055 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed5282-907c-4afc-9ebe-525ed80b7657 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:47.056338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed5282-907c-4afc-9ebe-525ed80b7657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:47.056611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed5282-907c-4afc-9ebe-525ed80b7657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:47.057317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed5282-907c-4afc-9ebe-525ed80b7657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:47.059200 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed5282-907c-4afc-9ebe-525ed80b7657 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:47.168799 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1236/2442] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:41:46 2022] POST /identity/v3/projects => generated 351 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:41:47.172788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa3a37-9dc1-4927-ba1c-c6ce943c0d13 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:47.183456 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa3a37-9dc1-4927-ba1c-c6ce943c0d13 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:47.238667 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83817986-25bb-4848-80a3-ebd4abb98237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:47.239338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83817986-25bb-4848-80a3-ebd4abb98237 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:47.239654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83817986-25bb-4848-80a3-ebd4abb98237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:47.239937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83817986-25bb-4848-80a3-ebd4abb98237 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:47.240622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83817986-25bb-4848-80a3-ebd4abb98237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:47.242546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83817986-25bb-4848-80a3-ebd4abb98237 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:47.344974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1207/2443] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:41:47 2022] POST /identity/v3/users => generated 282 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:41:47.349224 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed5282-907c-4afc-9ebe-525ed80b7657 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:47.360619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed5282-907c-4afc-9ebe-525ed80b7657 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:47.365465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452389c3-f4c5-4cf3-8528-eeda0c07ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:47.372498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452389c3-f4c5-4cf3-8528-eeda0c07ed2f admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:47.372498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452389c3-f4c5-4cf3-8528-eeda0c07ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:47.372498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452389c3-f4c5-4cf3-8528-eeda0c07ed2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:47.372498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452389c3-f4c5-4cf3-8528-eeda0c07ed2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:47.372498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452389c3-f4c5-4cf3-8528-eeda0c07ed2f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:47.379953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1237/2444] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:41:47.384136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83817986-25bb-4848-80a3-ebd4abb98237 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:47.396408 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83817986-25bb-4848-80a3-ebd4abb98237 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:47.401753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf1647-f988-48ae-a355-5e96f20f9ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:47.402649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf1647-f988-48ae-a355-5e96f20f9ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:47.402938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf1647-f988-48ae-a355-5e96f20f9ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:47.403214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf1647-f988-48ae-a355-5e96f20f9ae0 admin admin] PATH_INFO: `/v3/projects/34473a6e9d3e4caf9cbfccc1090b067d/users/5e9c811fda2449f1b47ab2c367adfade/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:47.419152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf1647-f988-48ae-a355-5e96f20f9ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34473a6e9d3e4caf9cbfccc1090b067d, user_id=5e9c811fda2449f1b47ab2c367adfade, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:47.421880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf1647-f988-48ae-a355-5e96f20f9ae0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:47.488483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1208/2445] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:41:47 2022] PUT /identity/v3/projects/34473a6e9d3e4caf9cbfccc1090b067d/users/5e9c811fda2449f1b47ab2c367adfade/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:41:47.492352 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452389c3-f4c5-4cf3-8528-eeda0c07ed2f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:47.503025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452389c3-f4c5-4cf3-8528-eeda0c07ed2f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:47.561912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb6d91-2872-4ab2-a18a-47cdf3ee984f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:47.562721 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb6d91-2872-4ab2-a18a-47cdf3ee984f admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:47.563000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb6d91-2872-4ab2-a18a-47cdf3ee984f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:47.563275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb6d91-2872-4ab2-a18a-47cdf3ee984f admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:47.564028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb6d91-2872-4ab2-a18a-47cdf3ee984f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:47.565618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb6d91-2872-4ab2-a18a-47cdf3ee984f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:47.576164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1238/2446] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:41:47.580531 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf1647-f988-48ae-a355-5e96f20f9ae0 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:47.591491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf1647-f988-48ae-a355-5e96f20f9ae0 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:47.596188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c176d5-4dda-4c0a-8b9b-25bc04b691a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:47.597073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c176d5-4dda-4c0a-8b9b-25bc04b691a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:47.597360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c176d5-4dda-4c0a-8b9b-25bc04b691a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:47.597643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c176d5-4dda-4c0a-8b9b-25bc04b691a1 admin admin] PATH_INFO: `/v3/projects/34473a6e9d3e4caf9cbfccc1090b067d/users/5e9c811fda2449f1b47ab2c367adfade/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:47.599656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c176d5-4dda-4c0a-8b9b-25bc04b691a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34473a6e9d3e4caf9cbfccc1090b067d, user_id=5e9c811fda2449f1b47ab2c367adfade, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:47.602321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c176d5-4dda-4c0a-8b9b-25bc04b691a1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:47.612493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1209/2447] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:41:47 2022] PUT /identity/v3/projects/34473a6e9d3e4caf9cbfccc1090b067d/users/5e9c811fda2449f1b47ab2c367adfade/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:41:47.623918 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d746b0-e701-497b-aa0c-54274e48e227 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:47.624212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d746b0-e701-497b-aa0c-54274e48e227 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:47.624495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d746b0-e701-497b-aa0c-54274e48e227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:47.650816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-49d746b0-e701-497b-aa0c-54274e48e227 None None] MFA Rules not processed for user `5e9c811fda2449f1b47ab2c367adfade`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:41:47.652060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-49d746b0-e701-497b-aa0c-54274e48e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:47.669921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:41:47.718389 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1239/2448] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:41:47 2022] POST /identity/v3/auth/tokens => generated 3285 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:41:47.730142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c176d5-4dda-4c0a-8b9b-25bc04b691a1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:47.731956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b9c176d5-4dda-4c0a-8b9b-25bc04b691a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:47.754440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:41:47.789341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c176d5-4dda-4c0a-8b9b-25bc04b691a1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:47.793861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb4852-ce9e-4d33-8d63-a83fb649e267 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:47.794521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb4852-ce9e-4d33-8d63-a83fb649e267 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:47.794804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb4852-ce9e-4d33-8d63-a83fb649e267 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:47.795081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb4852-ce9e-4d33-8d63-a83fb649e267 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:47.804154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb4852-ce9e-4d33-8d63-a83fb649e267 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:47.805574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb4852-ce9e-4d33-8d63-a83fb649e267 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:47.815331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1210/2449] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:41:47 2022] GET /identity/v3/auth/tokens => generated 3285 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:41:48.035008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d746b0-e701-497b-aa0c-54274e48e227 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:48.045481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d746b0-e701-497b-aa0c-54274e48e227 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:48.049345 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b70e5f-d663-467e-b99b-f44892ce59ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:48.049988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b70e5f-d663-467e-b99b-f44892ce59ad service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:48.050294 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b70e5f-d663-467e-b99b-f44892ce59ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:48.050615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b70e5f-d663-467e-b99b-f44892ce59ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:48.105930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b70e5f-d663-467e-b99b-f44892ce59ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:48.107472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b70e5f-d663-467e-b99b-f44892ce59ad service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:48.116279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1240/2450] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:41:48 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:41:48.126218 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb4852-ce9e-4d33-8d63-a83fb649e267 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:48.127622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-dabb4852-ce9e-4d33-8d63-a83fb649e267 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:48.202665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb4852-ce9e-4d33-8d63-a83fb649e267 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:48.208195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490cf41-fe0b-4d09-b685-a588f4fc865b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:48.208834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490cf41-fe0b-4d09-b685-a588f4fc865b service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:48.209114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490cf41-fe0b-4d09-b685-a588f4fc865b service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:48.209389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490cf41-fe0b-4d09-b685-a588f4fc865b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:48.211261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5490cf41-fe0b-4d09-b685-a588f4fc865b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:48.269809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490cf41-fe0b-4d09-b685-a588f4fc865b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:48.271422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490cf41-fe0b-4d09-b685-a588f4fc865b service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:48.281033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1211/2451] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:41:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:41:51.581593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b70e5f-d663-467e-b99b-f44892ce59ad service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:41:51.583832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-33b70e5f-d663-467e-b99b-f44892ce59ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:51.675576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b70e5f-d663-467e-b99b-f44892ce59ad service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:41:51.680163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:41:51.680952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:41:51.681546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:41:51.686212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:41:51.686212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:41:51.696720 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:41:51.698196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:41:51.726916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1241/2452] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:41:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:41:58.276533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=73 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:00.138661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5714d0a-cb04-4119-ac11-960a1e0be19f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:00.148119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5714d0a-cb04-4119-ac11-960a1e0be19f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:00.148119 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5714d0a-cb04-4119-ac11-960a1e0be19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:00.156929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-c5714d0a-cb04-4119-ac11-960a1e0be19f None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:00.158132 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c5714d0a-cb04-4119-ac11-960a1e0be19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:00.212872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1212/2453] 10.222.0.64 () {58 vars in 967 bytes} [Mon Jun 27 09:42:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:00.376972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:00.378425 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:00.432276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:00.447365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5d17c-da97-4762-a589-b2afae5123fd service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:00.453197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8d4dc-1a5d-4040-a322-9e8d0ac12592 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:00.453854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8d4dc-1a5d-4040-a322-9e8d0ac12592 service swift] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:00.454137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8d4dc-1a5d-4040-a322-9e8d0ac12592 service swift] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:00.454415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8d4dc-1a5d-4040-a322-9e8d0ac12592 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:00.462656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8d4dc-1a5d-4040-a322-9e8d0ac12592 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:00.464727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d8d4dc-1a5d-4040-a322-9e8d0ac12592 service swift] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:00.473153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1242/2454] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:42:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 09:42:06.788557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5714d0a-cb04-4119-ac11-960a1e0be19f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:06.874775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5714d0a-cb04-4119-ac11-960a1e0be19f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:06.881728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:06.882668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:06.882954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:06.883230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] PATH_INFO: `/v3/users/23094d07d73f425398caa2499bd816ef` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:06.884728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23094d07d73f425398caa2499bd816ef)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:06.887176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:07.083389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23094d07d73f425398caa2499bd816ef'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:07.084472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23094d07d73f425398caa2499bd816ef'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:07.093486 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23094d07d73f425398caa2499bd816ef'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:07.098100 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23094d07d73f425398caa2499bd816ef'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:07.111767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23094d07d73f425398caa2499bd816ef'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:07.206904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1213/2455] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:06 2022] DELETE /identity/v3/users/23094d07d73f425398caa2499bd816ef => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:07.208908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49805f9e-d622-40e9-a140-0d906470b7a5 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:07.209834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49805f9e-d622-40e9-a140-0d906470b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:07.210337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49805f9e-d622-40e9-a140-0d906470b7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:07.229707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-49805f9e-d622-40e9-a140-0d906470b7a5 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:07.231226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-49805f9e-d622-40e9-a140-0d906470b7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:07.314839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1243/2456] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:07.330856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:07.332656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:07.364476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:07.395452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eba4b4-a221-4a0a-8570-29b393ada6f3 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:07.399319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84265f5c-f058-4c31-9d3c-51adf72b6f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:07.400015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84265f5c-f058-4c31-9d3c-51adf72b6f14 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:07.400301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84265f5c-f058-4c31-9d3c-51adf72b6f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:07.400579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84265f5c-f058-4c31-9d3c-51adf72b6f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:07.411340 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84265f5c-f058-4c31-9d3c-51adf72b6f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:07.413434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84265f5c-f058-4c31-9d3c-51adf72b6f14 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:07.424731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1214/2457] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:07.940645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49805f9e-d622-40e9-a140-0d906470b7a5 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:07.942541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-49805f9e-d622-40e9-a140-0d906470b7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:07.962243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49805f9e-d622-40e9-a140-0d906470b7a5 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:07.968417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:07.969970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:07.970728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:07.971185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] PATH_INFO: `/v3/projects/e16d7489b72b41ce95a200ea6bf7ab2c` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:07.972749 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e16d7489b72b41ce95a200ea6bf7ab2c)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:07.974872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:08.110197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e16d7489b72b41ce95a200ea6bf7ab2c', 'request_id': 'req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.178202 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e16d7489b72b41ce95a200ea6bf7ab2c', 'request_id': 'req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.185023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e16d7489b72b41ce95a200ea6bf7ab2c', 'request_id': 'req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.199247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] The token cache is being invalidate because project e16d7489b72b41ce95a200ea6bf7ab2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:08.199727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.200533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.202104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1244/2458] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:07 2022] DELETE /identity/v3/projects/e16d7489b72b41ce95a200ea6bf7ab2c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:08.206737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84265f5c-f058-4c31-9d3c-51adf72b6f14 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:08.217513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84265f5c-f058-4c31-9d3c-51adf72b6f14 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:08.274272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:08.275276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:08.275595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:08.281291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] PATH_INFO: `/v3/users/5e9c811fda2449f1b47ab2c367adfade` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:08.281291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e9c811fda2449f1b47ab2c367adfade)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:08.281291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:08.492393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e9c811fda2449f1b47ab2c367adfade'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.493241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e9c811fda2449f1b47ab2c367adfade'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.501847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e9c811fda2449f1b47ab2c367adfade'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.502567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e9c811fda2449f1b47ab2c367adfade'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.516623 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e9c811fda2449f1b47ab2c367adfade'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:08.597697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1215/2459] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:08 2022] DELETE /identity/v3/users/5e9c811fda2449f1b47ab2c367adfade => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:08.926630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:08.929399 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:09.029245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d783572-4d2e-4f04-99f6-0e4c6a839e1d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:09.042180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:09.087087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:09.092592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:09.127324 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:09.128823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:09.129308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:09.130131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:09.136123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:09.151448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:09.155312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:09.183361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1245/2460] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:09.184421 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:09.189039 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:09.210040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:09.247142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226950b5-4d56-4182-8bb2-f02be935b70d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:09.263879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:09.265053 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:09.265400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:09.265677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] PATH_INFO: `/v3/users/5778103c646049f5944934e775cba9da` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:09.266960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5778103c646049f5944934e775cba9da)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:09.269015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:09.282147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52619e6c-1331-4cee-bd95-b26d391507e2 service placement] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:09.294118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:09.294996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:09.295282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:09.308771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] PATH_INFO: `/v3/projects/34473a6e9d3e4caf9cbfccc1090b067d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:09.308771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=34473a6e9d3e4caf9cbfccc1090b067d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:09.308771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:09.513435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34473a6e9d3e4caf9cbfccc1090b067d', 'request_id': 'req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.514660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5778103c646049f5944934e775cba9da'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.515325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5778103c646049f5944934e775cba9da'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.522247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5778103c646049f5944934e775cba9da'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.526469 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5778103c646049f5944934e775cba9da'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.535887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9b57cd70-46c2-4940-a94c-593937718d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5778103c646049f5944934e775cba9da'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.578015 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34473a6e9d3e4caf9cbfccc1090b067d', 'request_id': 'req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.585369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34473a6e9d3e4caf9cbfccc1090b067d', 'request_id': 'req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.601516 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] The token cache is being invalidate because project 34473a6e9d3e4caf9cbfccc1090b067d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:09.601807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.602504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a2a31ad4-e336-49a2-82ab-cc86442a1cdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:09.605206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1246/2461] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:09 2022] DELETE /identity/v3/projects/34473a6e9d3e4caf9cbfccc1090b067d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:09.609849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3ab3b-980a-4d36-b672-d40c04cf433b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:09.610141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3ab3b-980a-4d36-b672-d40c04cf433b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:09.610418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3ab3b-980a-4d36-b672-d40c04cf433b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:09.624315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1216/2462] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:09 2022] DELETE /identity/v3/users/5778103c646049f5944934e775cba9da => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:09.628151 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd78d8e-4640-4a47-8b06-2d50d2fa852b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:09.628618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd78d8e-4640-4a47-8b06-2d50d2fa852b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:09.629091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd78d8e-4640-4a47-8b06-2d50d2fa852b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:09.632393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-40c3ab3b-980a-4d36-b672-d40c04cf433b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:09.635166 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-40c3ab3b-980a-4d36-b672-d40c04cf433b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:09.653209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1fd78d8e-4640-4a47-8b06-2d50d2fa852b None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:09.655074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1fd78d8e-4640-4a47-8b06-2d50d2fa852b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:09.673648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:09.731043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1247/2463] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:42:09.738930 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2944c-00dc-4aee-93ce-bf27466bb89b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:09.739942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2944c-00dc-4aee-93ce-bf27466bb89b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:09.740231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2944c-00dc-4aee-93ce-bf27466bb89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:09.757225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1217/2464] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:42:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:09.764755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-22a2944c-00dc-4aee-93ce-bf27466bb89b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:09.766323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-22a2944c-00dc-4aee-93ce-bf27466bb89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:09.775514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd78d8e-4640-4a47-8b06-2d50d2fa852b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:09.779576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1248/2465] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:09.780626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1fd78d8e-4640-4a47-8b06-2d50d2fa852b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:09.781794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2944c-00dc-4aee-93ce-bf27466bb89b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:09.795446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2944c-00dc-4aee-93ce-bf27466bb89b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:09.803593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad7e11-3369-4528-ab94-12e92b777dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:09.804327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad7e11-3369-4528-ab94-12e92b777dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:09.805136 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad7e11-3369-4528-ab94-12e92b777dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:09.805392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad7e11-3369-4528-ab94-12e92b777dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:09.806117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad7e11-3369-4528-ab94-12e92b777dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:42:09.806725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad7e11-3369-4528-ab94-12e92b777dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:09.808542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad7e11-3369-4528-ab94-12e92b777dbc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:09.827603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1249/2466] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:42:09.833549 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad7e11-3369-4528-ab94-12e92b777dbc admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:09.842934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad7e11-3369-4528-ab94-12e92b777dbc admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:09.848505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cb53d-9559-4360-b6d3-a5f0b72ec623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:09.849177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cb53d-9559-4360-b6d3-a5f0b72ec623 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:09.849470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cb53d-9559-4360-b6d3-a5f0b72ec623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:09.849753 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cb53d-9559-4360-b6d3-a5f0b72ec623 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:09.850474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cb53d-9559-4360-b6d3-a5f0b72ec623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:09.852531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cb53d-9559-4360-b6d3-a5f0b72ec623 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:09.868466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd78d8e-4640-4a47-8b06-2d50d2fa852b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:09.874461 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cee37d-77d1-44de-8f81-48a57e7ff54b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:09.875128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cee37d-77d1-44de-8f81-48a57e7ff54b service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:09.875410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cee37d-77d1-44de-8f81-48a57e7ff54b service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:09.875718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cee37d-77d1-44de-8f81-48a57e7ff54b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:09.885138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cee37d-77d1-44de-8f81-48a57e7ff54b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:09.887322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cee37d-77d1-44de-8f81-48a57e7ff54b service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:09.893448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:09.897387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1218/2467] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:42:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:42:09.943921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1250/2468] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:09 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:42:09.948770 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cee37d-77d1-44de-8f81-48a57e7ff54b service swift] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:09.959582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cee37d-77d1-44de-8f81-48a57e7ff54b service swift] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:10.022544 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0e543-2f1f-4d3d-a300-1b27a039d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:10.023382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0e543-2f1f-4d3d-a300-1b27a039d279 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:10.023803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0e543-2f1f-4d3d-a300-1b27a039d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:10.024188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0e543-2f1f-4d3d-a300-1b27a039d279 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:10.025225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0e543-2f1f-4d3d-a300-1b27a039d279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:10.028203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0e543-2f1f-4d3d-a300-1b27a039d279 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:10.140159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1219/2469] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:09 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:42:10.145182 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cb53d-9559-4360-b6d3-a5f0b72ec623 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:10.156958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cb53d-9559-4360-b6d3-a5f0b72ec623 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:10.162172 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b923a0-4f28-4307-a6e3-9620ce49adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:10.163057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b923a0-4f28-4307-a6e3-9620ce49adf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:10.163343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b923a0-4f28-4307-a6e3-9620ce49adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:10.163651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b923a0-4f28-4307-a6e3-9620ce49adf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:10.164424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b923a0-4f28-4307-a6e3-9620ce49adf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:10.166102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b923a0-4f28-4307-a6e3-9620ce49adf8 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:10.185096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1251/2470] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:10.186300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0e543-2f1f-4d3d-a300-1b27a039d279 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:10.202017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b923a0-4f28-4307-a6e3-9620ce49adf8 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:10.208708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0e543-2f1f-4d3d-a300-1b27a039d279 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:10.211329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b0b923a0-4f28-4307-a6e3-9620ce49adf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:10.215270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd2d30-0447-482d-a420-c112b7d2908f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:10.217037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd2d30-0447-482d-a420-c112b7d2908f admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:10.217753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd2d30-0447-482d-a420-c112b7d2908f admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:10.218432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd2d30-0447-482d-a420-c112b7d2908f admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:10.219721 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd2d30-0447-482d-a420-c112b7d2908f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:10.221566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd2d30-0447-482d-a420-c112b7d2908f admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:10.243266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b923a0-4f28-4307-a6e3-9620ce49adf8 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:10.254141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:10.255940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:10.256342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1220/2471] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:42:10.256732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:10.259047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] PATH_INFO: `/v3/projects/55f89ed92a124e2c8a274975bc0935ba` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:10.259163 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd2d30-0447-482d-a420-c112b7d2908f admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:10.259277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f89ed92a124e2c8a274975bc0935ba)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:10.262974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:10.282216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd2d30-0447-482d-a420-c112b7d2908f admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:10.300753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eff872-b753-4686-ba25-552b0a6bd13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:10.301904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eff872-b753-4686-ba25-552b0a6bd13e admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:10.302348 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eff872-b753-4686-ba25-552b0a6bd13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:10.302803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eff872-b753-4686-ba25-552b0a6bd13e admin admin] PATH_INFO: `/v3/projects/3a5199b723bb41d58e3eb5c35bfbdc57/users/afa7b1d31ad647b2a9c3d20a6e555a9c/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:10.340670 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eff872-b753-4686-ba25-552b0a6bd13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5199b723bb41d58e3eb5c35bfbdc57, user_id=afa7b1d31ad647b2a9c3d20a6e555a9c, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:10.343729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eff872-b753-4686-ba25-552b0a6bd13e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:10.457842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f89ed92a124e2c8a274975bc0935ba', 'request_id': 'req-70aae622-3335-48b3-8727-30aa68c1ff5b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.461393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1221/2472] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:10 2022] PUT /identity/v3/projects/3a5199b723bb41d58e3eb5c35bfbdc57/users/afa7b1d31ad647b2a9c3d20a6e555a9c/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:10.467161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6d0d4-8872-4747-8fa9-b6a4b9dd7d6a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:10.467921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6d0d4-8872-4747-8fa9-b6a4b9dd7d6a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:10.468395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6d0d4-8872-4747-8fa9-b6a4b9dd7d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:10.523809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-38b6d0d4-8872-4747-8fa9-b6a4b9dd7d6a None None] MFA Rules not processed for user `afa7b1d31ad647b2a9c3d20a6e555a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:10.525863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-38b6d0d4-8872-4747-8fa9-b6a4b9dd7d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:10.559585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f89ed92a124e2c8a274975bc0935ba', 'request_id': 'req-70aae622-3335-48b3-8727-30aa68c1ff5b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.570661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f89ed92a124e2c8a274975bc0935ba', 'request_id': 'req-70aae622-3335-48b3-8727-30aa68c1ff5b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.585088 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] The token cache is being invalidate because project 55f89ed92a124e2c8a274975bc0935ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:10.587143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.587143 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.588579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1252/2473] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:10 2022] DELETE /identity/v3/projects/55f89ed92a124e2c8a274975bc0935ba => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:10.592598 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:10.594443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:10.610719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1222/2474] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:10.621128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6d0d4-8872-4747-8fa9-b6a4b9dd7d6a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:10.622618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-38b6d0d4-8872-4747-8fa9-b6a4b9dd7d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:10.680910 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:10.691793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aae622-3335-48b3-8727-30aa68c1ff5b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:10.705096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:10.705994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:10.706277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:10.706557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] PATH_INFO: `/v3/users/1dcca7ce110547b8a6bfdf4a84f6f220` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:10.706984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6d0d4-8872-4747-8fa9-b6a4b9dd7d6a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:10.709283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dcca7ce110547b8a6bfdf4a84f6f220)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:10.709979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:10.712194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2baf75-1d5a-43aa-b1d2-cda96c2d8023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:10.712920 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2baf75-1d5a-43aa-b1d2-cda96c2d8023 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:10.713206 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2baf75-1d5a-43aa-b1d2-cda96c2d8023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:10.714154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2baf75-1d5a-43aa-b1d2-cda96c2d8023 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:10.714876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2baf75-1d5a-43aa-b1d2-cda96c2d8023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:10.716148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:10.717640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2baf75-1d5a-43aa-b1d2-cda96c2d8023 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:10.810045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1223/2475] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:10 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:42:10.815950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2baf75-1d5a-43aa-b1d2-cda96c2d8023 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:10.884614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dcca7ce110547b8a6bfdf4a84f6f220'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.897454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2baf75-1d5a-43aa-b1d2-cda96c2d8023 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:10.906597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b32ae0-9172-4f22-bbaa-b4184a3cb8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:10.906597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b32ae0-9172-4f22-bbaa-b4184a3cb8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:10.906597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b32ae0-9172-4f22-bbaa-b4184a3cb8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:10.906597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b32ae0-9172-4f22-bbaa-b4184a3cb8b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:10.907022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b32ae0-9172-4f22-bbaa-b4184a3cb8b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:10.907959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b32ae0-9172-4f22-bbaa-b4184a3cb8b6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:10.954095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dcca7ce110547b8a6bfdf4a84f6f220'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.955028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dcca7ce110547b8a6bfdf4a84f6f220'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.969221 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dcca7ce110547b8a6bfdf4a84f6f220'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.969955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dcca7ce110547b8a6bfdf4a84f6f220'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:10.993514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1224/2477] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:10 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:10.999489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1253/2477] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:10 2022] DELETE /identity/v3/users/1dcca7ce110547b8a6bfdf4a84f6f220 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:11.009525 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b32ae0-9172-4f22-bbaa-b4184a3cb8b6 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:11.018179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-96b32ae0-9172-4f22-bbaa-b4184a3cb8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:11.045237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b32ae0-9172-4f22-bbaa-b4184a3cb8b6 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:11.092177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:11.093833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:11.199509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f334b9-5b0f-4e06-93dd-fdcb3dec769c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:11.200942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f334b9-5b0f-4e06-93dd-fdcb3dec769c admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:11.200942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f334b9-5b0f-4e06-93dd-fdcb3dec769c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:11.201723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f334b9-5b0f-4e06-93dd-fdcb3dec769c admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:11.202660 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f334b9-5b0f-4e06-93dd-fdcb3dec769c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:11.204872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f334b9-5b0f-4e06-93dd-fdcb3dec769c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:11.219029 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1225/2478] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:11.231326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f334b9-5b0f-4e06-93dd-fdcb3dec769c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:11.237347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d247e-bbd9-42b3-b406-63ac8f904214 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:11.246370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f334b9-5b0f-4e06-93dd-fdcb3dec769c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:11.248842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:11.249648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:11.249936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:11.250211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:11.252563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c2791-126c-4df2-b2f4-05976a692cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:11.253997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c2791-126c-4df2-b2f4-05976a692cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:11.254402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c2791-126c-4df2-b2f4-05976a692cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:11.255009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c2791-126c-4df2-b2f4-05976a692cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:11.255881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c2791-126c-4df2-b2f4-05976a692cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:11.258390 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c2791-126c-4df2-b2f4-05976a692cd2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:11.260956 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:11.283598 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1226/2479] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:11.290253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:11.290253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:11.294067 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c2791-126c-4df2-b2f4-05976a692cd2 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:11.302973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1254/2480] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:42:11.312084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c2791-126c-4df2-b2f4-05976a692cd2 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:11.318685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb9709-3170-4142-b4e8-15cbe1b8c439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:11.320098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb9709-3170-4142-b4e8-15cbe1b8c439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:11.320098 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb9709-3170-4142-b4e8-15cbe1b8c439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:11.320684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb9709-3170-4142-b4e8-15cbe1b8c439 admin admin] PATH_INFO: `/v3/projects/16f32265d5a749408f55592de42c8eb1/users/bf8757b4148e41bc8050929908484305/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:11.340290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb9709-3170-4142-b4e8-15cbe1b8c439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f32265d5a749408f55592de42c8eb1, user_id=bf8757b4148e41bc8050929908484305, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:11.343186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb9709-3170-4142-b4e8-15cbe1b8c439 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:11.374673 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:11.376203 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:11.390912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26274d69-5567-441d-b88d-8b1bd1ea6f1c service nova] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:11.396038 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:11.397366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:11.397819 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:11.398253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] PATH_INFO: `/v3/projects/9550c13ccce1494b820141ec0acc768e` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:11.400338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9550c13ccce1494b820141ec0acc768e)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:11.402321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:11.453219 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1227/2481] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:11 2022] PUT /identity/v3/projects/16f32265d5a749408f55592de42c8eb1/users/bf8757b4148e41bc8050929908484305/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:11.463186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce02227-f56c-44c8-b19b-cd85fef25b66 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:11.463639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce02227-f56c-44c8-b19b-cd85fef25b66 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:11.463989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce02227-f56c-44c8-b19b-cd85fef25b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:11.497208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-2ce02227-f56c-44c8-b19b-cd85fef25b66 None None] MFA Rules not processed for user `bf8757b4148e41bc8050929908484305`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:11.498823 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2ce02227-f56c-44c8-b19b-cd85fef25b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:11.512962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:11.528676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9550c13ccce1494b820141ec0acc768e', 'request_id': 'req-50f0e10e-83f2-4a62-a602-c1abc34bebe4'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:11.602536 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1228/2482] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:11.607736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce02227-f56c-44c8-b19b-cd85fef25b66 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:11.608429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9550c13ccce1494b820141ec0acc768e', 'request_id': 'req-50f0e10e-83f2-4a62-a602-c1abc34bebe4'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:11.618443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:11.621795 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9550c13ccce1494b820141ec0acc768e', 'request_id': 'req-50f0e10e-83f2-4a62-a602-c1abc34bebe4'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:11.638076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] The token cache is being invalidate because project 9550c13ccce1494b820141ec0acc768e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:11.638759 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:11.639644 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:11.641900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1255/2483] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:11 2022] DELETE /identity/v3/projects/9550c13ccce1494b820141ec0acc768e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:11.773370 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce02227-f56c-44c8-b19b-cd85fef25b66 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:11.868942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67904e3-c79c-4e9f-8f64-ad9ad03084d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:11.870309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67904e3-c79c-4e9f-8f64-ad9ad03084d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:11.870585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67904e3-c79c-4e9f-8f64-ad9ad03084d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:11.870973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67904e3-c79c-4e9f-8f64-ad9ad03084d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:11.871728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67904e3-c79c-4e9f-8f64-ad9ad03084d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:11.873731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67904e3-c79c-4e9f-8f64-ad9ad03084d2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:11.966531 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1229/2484] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:11 2022] POST /identity/v3/projects => generated 383 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:11.968832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:11.969310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:11.992580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0e10e-83f2-4a62-a602-c1abc34bebe4 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:12.060219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66e93d-de88-4423-8cb6-f162ba38faf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:12.063100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66e93d-de88-4423-8cb6-f162ba38faf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:12.063100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66e93d-de88-4423-8cb6-f162ba38faf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:12.063100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66e93d-de88-4423-8cb6-f162ba38faf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:12.063100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66e93d-de88-4423-8cb6-f162ba38faf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:12.064111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66e93d-de88-4423-8cb6-f162ba38faf0 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:12.143376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1256/2485] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:11 2022] POST /identity/v3/users => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:12.147367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67904e3-c79c-4e9f-8f64-ad9ad03084d2 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:12.165760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67904e3-c79c-4e9f-8f64-ad9ad03084d2 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:12.171521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55877e57-492c-4e96-89f2-783535de399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:12.172332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55877e57-492c-4e96-89f2-783535de399b admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:12.172622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55877e57-492c-4e96-89f2-783535de399b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:12.172991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55877e57-492c-4e96-89f2-783535de399b admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:12.173759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55877e57-492c-4e96-89f2-783535de399b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:12.175397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55877e57-492c-4e96-89f2-783535de399b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:12.189747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1230/2486] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:12.194060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66e93d-de88-4423-8cb6-f162ba38faf0 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:12.206149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66e93d-de88-4423-8cb6-f162ba38faf0 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:12.213008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1030c3-0505-47b1-b0cf-1ab115c02127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:12.214043 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1030c3-0505-47b1-b0cf-1ab115c02127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:12.214210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1030c3-0505-47b1-b0cf-1ab115c02127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:12.214493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1030c3-0505-47b1-b0cf-1ab115c02127 admin admin] PATH_INFO: `/v3/projects/6270e4b669d34601ab764c2dac4d0c07/users/711edf03d121404abecf4c74040800cc/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:12.244472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1030c3-0505-47b1-b0cf-1ab115c02127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6270e4b669d34601ab764c2dac4d0c07, user_id=711edf03d121404abecf4c74040800cc, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:12.248038 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1030c3-0505-47b1-b0cf-1ab115c02127 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:12.329395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1257/2487] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:12 2022] PUT /identity/v3/projects/6270e4b669d34601ab764c2dac4d0c07/users/711edf03d121404abecf4c74040800cc/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:12.330719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55877e57-492c-4e96-89f2-783535de399b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:12.341917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55877e57-492c-4e96-89f2-783535de399b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:12.379466 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:12.411410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b17499-1dbf-443f-88a8-7d7bd0ae262f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:12.412573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b17499-1dbf-443f-88a8-7d7bd0ae262f admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:12.413796 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b17499-1dbf-443f-88a8-7d7bd0ae262f admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:12.414075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b17499-1dbf-443f-88a8-7d7bd0ae262f admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:12.414806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b17499-1dbf-443f-88a8-7d7bd0ae262f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:12.416574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b17499-1dbf-443f-88a8-7d7bd0ae262f admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:12.428690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1231/2488] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:12.432823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1030c3-0505-47b1-b0cf-1ab115c02127 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:12.446001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1030c3-0505-47b1-b0cf-1ab115c02127 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:12.451100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efa774-022d-4ae6-8772-1ecacaea9c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:12.452162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efa774-022d-4ae6-8772-1ecacaea9c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:12.452553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efa774-022d-4ae6-8772-1ecacaea9c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:12.452896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efa774-022d-4ae6-8772-1ecacaea9c2f admin admin] PATH_INFO: `/v3/projects/6270e4b669d34601ab764c2dac4d0c07/users/711edf03d121404abecf4c74040800cc/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:12.454951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efa774-022d-4ae6-8772-1ecacaea9c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6270e4b669d34601ab764c2dac4d0c07, user_id=711edf03d121404abecf4c74040800cc, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:12.457969 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efa774-022d-4ae6-8772-1ecacaea9c2f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:12.471717 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1258/2489] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:12 2022] PUT /identity/v3/projects/6270e4b669d34601ab764c2dac4d0c07/users/711edf03d121404abecf4c74040800cc/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:12.484139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61db8f-38c2-409b-8bed-6359a364a06c None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:12.484825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61db8f-38c2-409b-8bed-6359a364a06c None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:12.485106 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61db8f-38c2-409b-8bed-6359a364a06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:12.517116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-db61db8f-38c2-409b-8bed-6359a364a06c None None] MFA Rules not processed for user `711edf03d121404abecf4c74040800cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:12.518358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-db61db8f-38c2-409b-8bed-6359a364a06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:12.601237 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1232/2490] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:12.615403 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efa774-022d-4ae6-8772-1ecacaea9c2f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:12.615403 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b8efa774-022d-4ae6-8772-1ecacaea9c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:12.681797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efa774-022d-4ae6-8772-1ecacaea9c2f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:12.686529 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4882-8608-4319-87d0-9ef7f66847cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:12.686760 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4882-8608-4319-87d0-9ef7f66847cd service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:12.687191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4882-8608-4319-87d0-9ef7f66847cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:12.687304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4882-8608-4319-87d0-9ef7f66847cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:12.701559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4882-8608-4319-87d0-9ef7f66847cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:12.704371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4882-8608-4319-87d0-9ef7f66847cd service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:12.720652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1259/2491] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:14.210493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61db8f-38c2-409b-8bed-6359a364a06c None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:14.211942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-db61db8f-38c2-409b-8bed-6359a364a06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:14.226449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61db8f-38c2-409b-8bed-6359a364a06c None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:14.231056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b458039-5a63-4911-b4c5-9fbe229ae412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:14.231749 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b458039-5a63-4911-b4c5-9fbe229ae412 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:14.232035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b458039-5a63-4911-b4c5-9fbe229ae412 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:14.232314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b458039-5a63-4911-b4c5-9fbe229ae412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:14.234121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5b458039-5a63-4911-b4c5-9fbe229ae412 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:14.310373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b458039-5a63-4911-b4c5-9fbe229ae412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:14.312054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b458039-5a63-4911-b4c5-9fbe229ae412 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:14.323581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1233/2492] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:19.798686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4882-8608-4319-87d0-9ef7f66847cd service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:19.809777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4882-8608-4319-87d0-9ef7f66847cd service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:19.817910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad9e34-f69d-462c-be3c-2331240e2629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:19.819132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad9e34-f69d-462c-be3c-2331240e2629 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:19.819609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad9e34-f69d-462c-be3c-2331240e2629 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:19.820002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad9e34-f69d-462c-be3c-2331240e2629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:19.822626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-caad9e34-f69d-462c-be3c-2331240e2629 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:19.881772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:19.906537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad9e34-f69d-462c-be3c-2331240e2629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:19.908824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad9e34-f69d-462c-be3c-2331240e2629 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:19.923520 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1260/2493] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:24.317554 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=77 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:24.430489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b458039-5a63-4911-b4c5-9fbe229ae412 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:24.493686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b458039-5a63-4911-b4c5-9fbe229ae412 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:24.498956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:24.499865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:24.500149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:24.500428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] PATH_INFO: `/v3/users/afa7b1d31ad647b2a9c3d20a6e555a9c` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:24.501637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa7b1d31ad647b2a9c3d20a6e555a9c)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:24.503609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:24.707244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa7b1d31ad647b2a9c3d20a6e555a9c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:24.707957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa7b1d31ad647b2a9c3d20a6e555a9c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:24.715406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa7b1d31ad647b2a9c3d20a6e555a9c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:24.716202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa7b1d31ad647b2a9c3d20a6e555a9c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:24.728652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa7b1d31ad647b2a9c3d20a6e555a9c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:24.853456 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1234/2494] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:24 2022] DELETE /identity/v3/users/afa7b1d31ad647b2a9c3d20a6e555a9c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:24.858577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf1a6c-0fe6-4ad6-b515-66dc0907ec8a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:24.859085 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf1a6c-0fe6-4ad6-b515-66dc0907ec8a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:24.859183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf1a6c-0fe6-4ad6-b515-66dc0907ec8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:24.876635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3bbf1a6c-0fe6-4ad6-b515-66dc0907ec8a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:24.877854 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3bbf1a6c-0fe6-4ad6-b515-66dc0907ec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:24.939451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1261/2495] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:24.949429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:24.951120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:25.017702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e1ccd-5f08-4325-a1e9-94387bf224a4 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:25.028309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed8d9b-0d4e-4819-b633-2b2d5a3b242c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:25.028992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed8d9b-0d4e-4819-b633-2b2d5a3b242c service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:25.029273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed8d9b-0d4e-4819-b633-2b2d5a3b242c service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:25.029558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed8d9b-0d4e-4819-b633-2b2d5a3b242c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:25.041012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed8d9b-0d4e-4819-b633-2b2d5a3b242c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:25.043401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed8d9b-0d4e-4819-b633-2b2d5a3b242c service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:25.053852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1235/2496] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:25.511703 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf1a6c-0fe6-4ad6-b515-66dc0907ec8a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:25.514115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3bbf1a6c-0fe6-4ad6-b515-66dc0907ec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:25.531149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf1a6c-0fe6-4ad6-b515-66dc0907ec8a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:25.537767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:25.546452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:25.546452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:25.546452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] PATH_INFO: `/v3/projects/3a5199b723bb41d58e3eb5c35bfbdc57` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:25.546452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5199b723bb41d58e3eb5c35bfbdc57)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:25.546452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:25.663912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5199b723bb41d58e3eb5c35bfbdc57', 'request_id': 'req-e1f583f5-560c-44d5-ab8b-d45c2552030b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:25.779394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5199b723bb41d58e3eb5c35bfbdc57', 'request_id': 'req-e1f583f5-560c-44d5-ab8b-d45c2552030b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:25.786387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5199b723bb41d58e3eb5c35bfbdc57', 'request_id': 'req-e1f583f5-560c-44d5-ab8b-d45c2552030b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:25.798605 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] The token cache is being invalidate because project 3a5199b723bb41d58e3eb5c35bfbdc57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:25.799027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:25.799730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e1f583f5-560c-44d5-ab8b-d45c2552030b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:25.801907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1262/2497] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:25 2022] DELETE /identity/v3/projects/3a5199b723bb41d58e3eb5c35bfbdc57 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:25.805981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed8d9b-0d4e-4819-b633-2b2d5a3b242c service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:25.818859 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed8d9b-0d4e-4819-b633-2b2d5a3b242c service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:25.837961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:25.876957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:25.886529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:25.886529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:25.886529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] PATH_INFO: `/v3/users/bf8757b4148e41bc8050929908484305` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:25.886529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf8757b4148e41bc8050929908484305)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:25.886529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:25.888941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca0ae3-aa9a-4d2f-9e83-b64932cd7843 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:25.889310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca0ae3-aa9a-4d2f-9e83-b64932cd7843 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:25.889417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca0ae3-aa9a-4d2f-9e83-b64932cd7843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:25.908447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-03ca0ae3-aa9a-4d2f-9e83-b64932cd7843 None None] MFA Rules not processed for user `cc8a087e97424d2890952be33695378f`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:25.909676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-03ca0ae3-aa9a-4d2f-9e83-b64932cd7843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:25.966423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1263/2498] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:26.077633 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf8757b4148e41bc8050929908484305'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:26.077857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf8757b4148e41bc8050929908484305'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:26.084937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf8757b4148e41bc8050929908484305'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:26.085582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf8757b4148e41bc8050929908484305'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:26.102983 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf8757b4148e41bc8050929908484305'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:26.200058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1236/2499] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:25 2022] DELETE /identity/v3/users/bf8757b4148e41bc8050929908484305 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:26.281435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca0ae3-aa9a-4d2f-9e83-b64932cd7843 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:26.282958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-03ca0ae3-aa9a-4d2f-9e83-b64932cd7843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:26.302354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:26.346288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca0ae3-aa9a-4d2f-9e83-b64932cd7843 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:26.402798 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:26.404044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:26.404446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:26.404582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:26.406416 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:26.418829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:26.420340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:26.430443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1264/2500] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:26.780146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:26.781611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:26.852268 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf57250-7fb4-4e70-847e-bedcd95c56a2 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:26.857542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:26.858400 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:26.858674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:26.858942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] PATH_INFO: `/v3/projects/16f32265d5a749408f55592de42c8eb1` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:26.860142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f32265d5a749408f55592de42c8eb1)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:26.862038 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:26.923257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:26.925449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:26.968323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:26.970959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f32265d5a749408f55592de42c8eb1', 'request_id': 'req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:26.976715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f32265d5a749408f55592de42c8eb1', 'request_id': 'req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:26.983494 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f32265d5a749408f55592de42c8eb1', 'request_id': 'req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:27.017471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447867-7a19-4d36-9cc3-21f5e258e16f service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:27.094049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ddf00-afb9-4f17-969c-457f7ec9b796 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:27.095058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ddf00-afb9-4f17-969c-457f7ec9b796 service placement] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:27.095176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ddf00-afb9-4f17-969c-457f7ec9b796 service placement] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:27.095423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ddf00-afb9-4f17-969c-457f7ec9b796 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:27.097374 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ac5ddf00-afb9-4f17-969c-457f7ec9b796 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:27.106332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] The token cache is being invalidate because project 16f32265d5a749408f55592de42c8eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:27.106628 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:27.108048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:27.110476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1237/2501] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:26 2022] DELETE /identity/v3/projects/16f32265d5a749408f55592de42c8eb1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:27.116105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:27.116423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:27.116823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ddf00-afb9-4f17-969c-457f7ec9b796 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:27.118782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ddf00-afb9-4f17-969c-457f7ec9b796 service placement] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:27.135857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1265/2502] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:42:27.198644 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882d46c-4c29-455e-a6f7-6fddf1e7cdbb admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:27.206714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:27.206714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:27.206714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:27.206714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] PATH_INFO: `/v3/users/711edf03d121404abecf4c74040800cc` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:27.208750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=711edf03d121404abecf4c74040800cc)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:27.211711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:27.468839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '711edf03d121404abecf4c74040800cc'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:27.469532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '711edf03d121404abecf4c74040800cc'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:27.477616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '711edf03d121404abecf4c74040800cc'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:27.478383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '711edf03d121404abecf4c74040800cc'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:27.506681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-894b8c07-54c0-4f57-b74f-6f8c9fdf9410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '711edf03d121404abecf4c74040800cc'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:27.617895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1238/2503] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:27 2022] DELETE /identity/v3/users/711edf03d121404abecf4c74040800cc => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:28.098136 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ddf00-afb9-4f17-969c-457f7ec9b796 service placement] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:28.107844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:28.112371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ddf00-afb9-4f17-969c-457f7ec9b796 service placement] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:28.176648 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:28.177619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:28.177903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:28.178475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] PATH_INFO: `/v3/projects/6270e4b669d34601ab764c2dac4d0c07` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:28.179729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6270e4b669d34601ab764c2dac4d0c07)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:28.181670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:28.320663 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6270e4b669d34601ab764c2dac4d0c07', 'request_id': 'req-fdd3476e-d37c-4a5e-b68b-06234180fd7a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:28.427371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6270e4b669d34601ab764c2dac4d0c07', 'request_id': 'req-fdd3476e-d37c-4a5e-b68b-06234180fd7a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:28.434029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6270e4b669d34601ab764c2dac4d0c07', 'request_id': 'req-fdd3476e-d37c-4a5e-b68b-06234180fd7a'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:28.445269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] The token cache is being invalidate because project 6270e4b669d34601ab764c2dac4d0c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:28.445548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:28.446184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-fdd3476e-d37c-4a5e-b68b-06234180fd7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:28.447741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1266/2504] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:28 2022] DELETE /identity/v3/projects/6270e4b669d34601ab764c2dac4d0c07 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:28.454905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bb029-0454-4299-a04d-fab7d4eb99b0 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:28.455392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bb029-0454-4299-a04d-fab7d4eb99b0 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:28.455506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bb029-0454-4299-a04d-fab7d4eb99b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:28.476324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-2d8bb029-0454-4299-a04d-fab7d4eb99b0 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:28.477778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2d8bb029-0454-4299-a04d-fab7d4eb99b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:28.527116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:28.543628 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1239/2505] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:28.555408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792e551-282f-4576-bd4f-c89ef86a2f9f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:28.555723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792e551-282f-4576-bd4f-c89ef86a2f9f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:28.556001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792e551-282f-4576-bd4f-c89ef86a2f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:28.578463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-9792e551-282f-4576-bd4f-c89ef86a2f9f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:28.579804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9792e551-282f-4576-bd4f-c89ef86a2f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:28.587833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1267/2506] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:28.592872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bb029-0454-4299-a04d-fab7d4eb99b0 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:28.602933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bb029-0454-4299-a04d-fab7d4eb99b0 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:28.610845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea8654-ee17-4b33-876c-111e71fd1866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:28.611677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea8654-ee17-4b33-876c-111e71fd1866 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:28.612048 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea8654-ee17-4b33-876c-111e71fd1866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:28.612391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea8654-ee17-4b33-876c-111e71fd1866 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:28.613205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea8654-ee17-4b33-876c-111e71fd1866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:42:28.613569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea8654-ee17-4b33-876c-111e71fd1866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:28.615282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea8654-ee17-4b33-876c-111e71fd1866 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:28.628666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1240/2507] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:42:28.632843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792e551-282f-4576-bd4f-c89ef86a2f9f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:28.642629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792e551-282f-4576-bd4f-c89ef86a2f9f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:28.648372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee6d4a-91f3-408e-80a2-2e7de9647f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:28.649070 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee6d4a-91f3-408e-80a2-2e7de9647f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:28.649353 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee6d4a-91f3-408e-80a2-2e7de9647f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:28.649627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee6d4a-91f3-408e-80a2-2e7de9647f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:28.650322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee6d4a-91f3-408e-80a2-2e7de9647f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:28.652578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee6d4a-91f3-408e-80a2-2e7de9647f2c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:28.749808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1268/2508] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:28 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:42:28.753996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea8654-ee17-4b33-876c-111e71fd1866 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:28.764275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea8654-ee17-4b33-876c-111e71fd1866 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:28.821122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab79c7-7201-46e6-9eaa-83bd2c0a9e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:28.821821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab79c7-7201-46e6-9eaa-83bd2c0a9e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:28.822103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab79c7-7201-46e6-9eaa-83bd2c0a9e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:28.822381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab79c7-7201-46e6-9eaa-83bd2c0a9e52 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:28.823109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab79c7-7201-46e6-9eaa-83bd2c0a9e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:28.825063 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab79c7-7201-46e6-9eaa-83bd2c0a9e52 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:28.931304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1241/2509] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:28 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:42:28.935434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee6d4a-91f3-408e-80a2-2e7de9647f2c admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:28.946278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee6d4a-91f3-408e-80a2-2e7de9647f2c admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:28.951240 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aa543-575b-4b6c-874d-0ce85a97edab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:28.951958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aa543-575b-4b6c-874d-0ce85a97edab admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:28.952247 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aa543-575b-4b6c-874d-0ce85a97edab admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:28.952528 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aa543-575b-4b6c-874d-0ce85a97edab admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:28.953292 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8aa543-575b-4b6c-874d-0ce85a97edab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:28.954996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8aa543-575b-4b6c-874d-0ce85a97edab admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:28.965664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1269/2510] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:28.969654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab79c7-7201-46e6-9eaa-83bd2c0a9e52 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:28.980117 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab79c7-7201-46e6-9eaa-83bd2c0a9e52 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:28.984700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb1bb2-08ec-479c-b415-1762e4f870e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:28.985341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb1bb2-08ec-479c-b415-1762e4f870e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:28.985622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb1bb2-08ec-479c-b415-1762e4f870e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:28.985893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb1bb2-08ec-479c-b415-1762e4f870e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:28.986609 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb1bb2-08ec-479c-b415-1762e4f870e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:28.988220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb1bb2-08ec-479c-b415-1762e4f870e6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:29.002955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1242/2511] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:29.007046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aa543-575b-4b6c-874d-0ce85a97edab admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:29.017080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aa543-575b-4b6c-874d-0ce85a97edab admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:29.021801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eb5f7-f063-4033-b0aa-853467a33504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:29.022666 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eb5f7-f063-4033-b0aa-853467a33504 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:29.022951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eb5f7-f063-4033-b0aa-853467a33504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:29.023227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eb5f7-f063-4033-b0aa-853467a33504 admin admin] PATH_INFO: `/v3/projects/b0f8a4afc3754ac2a821fca7c9588c57/users/eaa48e567e724ad884873c39d25c33e2/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:29.038492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3eb5f7-f063-4033-b0aa-853467a33504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f8a4afc3754ac2a821fca7c9588c57, user_id=eaa48e567e724ad884873c39d25c33e2, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:29.041208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3eb5f7-f063-4033-b0aa-853467a33504 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:29.116625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1270/2512] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:29 2022] PUT /identity/v3/projects/b0f8a4afc3754ac2a821fca7c9588c57/users/eaa48e567e724ad884873c39d25c33e2/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:29.131157 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7d56a-e34c-4e6d-9d4d-35543b9dc229 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:29.131580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7d56a-e34c-4e6d-9d4d-35543b9dc229 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:29.131817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7d56a-e34c-4e6d-9d4d-35543b9dc229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:29.162731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a0d7d56a-e34c-4e6d-9d4d-35543b9dc229 None None] MFA Rules not processed for user `eaa48e567e724ad884873c39d25c33e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:29.164217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a0d7d56a-e34c-4e6d-9d4d-35543b9dc229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:29.233505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1243/2513] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:42:29.238271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eb5f7-f063-4033-b0aa-853467a33504 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:29.265205 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:29.307254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eb5f7-f063-4033-b0aa-853467a33504 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:29.311887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5fd79-5fa0-4c9f-91cc-a0f1420fb522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:29.312544 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5fd79-5fa0-4c9f-91cc-a0f1420fb522 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:29.312824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5fd79-5fa0-4c9f-91cc-a0f1420fb522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:29.313097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5fd79-5fa0-4c9f-91cc-a0f1420fb522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:29.313773 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5fd79-5fa0-4c9f-91cc-a0f1420fb522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:29.315775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5fd79-5fa0-4c9f-91cc-a0f1420fb522 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:29.393687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1271/2514] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:29 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:29.397917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7d56a-e34c-4e6d-9d4d-35543b9dc229 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:29.409280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7d56a-e34c-4e6d-9d4d-35543b9dc229 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:29.452372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:29.465671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baaee1-42b7-4ffa-bc42-c8505be4fea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:29.466377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baaee1-42b7-4ffa-bc42-c8505be4fea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:29.466663 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baaee1-42b7-4ffa-bc42-c8505be4fea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:29.466967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baaee1-42b7-4ffa-bc42-c8505be4fea2 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:29.467705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38baaee1-42b7-4ffa-bc42-c8505be4fea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:29.469753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38baaee1-42b7-4ffa-bc42-c8505be4fea2 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:29.587966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1244/2515] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:29 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:29.592534 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5fd79-5fa0-4c9f-91cc-a0f1420fb522 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:29.604023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5fd79-5fa0-4c9f-91cc-a0f1420fb522 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:29.609180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4f8ce-62ae-402d-93d2-ae8ac2a28aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:29.609843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4f8ce-62ae-402d-93d2-ae8ac2a28aff admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:29.610126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4f8ce-62ae-402d-93d2-ae8ac2a28aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:29.610429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4f8ce-62ae-402d-93d2-ae8ac2a28aff admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:29.612010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4f8ce-62ae-402d-93d2-ae8ac2a28aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:29.613726 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4f8ce-62ae-402d-93d2-ae8ac2a28aff admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:29.630649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1272/2516] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:29.631487 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baaee1-42b7-4ffa-bc42-c8505be4fea2 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:29.642987 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baaee1-42b7-4ffa-bc42-c8505be4fea2 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:29.648011 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a47738-55ab-4165-84f8-d818de051713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:29.648699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a47738-55ab-4165-84f8-d818de051713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:29.648981 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a47738-55ab-4165-84f8-d818de051713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:29.649355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a47738-55ab-4165-84f8-d818de051713 admin admin] PATH_INFO: `/v3/projects/fd6853316a884b3bb1b01197c43c2d9e/users/602ecb0aaf794ee4a89ffbad75584a01/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:29.670148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a47738-55ab-4165-84f8-d818de051713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6853316a884b3bb1b01197c43c2d9e, user_id=602ecb0aaf794ee4a89ffbad75584a01, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:29.673278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a47738-55ab-4165-84f8-d818de051713 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:29.763923 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1245/2517] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:29 2022] PUT /identity/v3/projects/fd6853316a884b3bb1b01197c43c2d9e/users/602ecb0aaf794ee4a89ffbad75584a01/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:29.768248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4f8ce-62ae-402d-93d2-ae8ac2a28aff admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:29.778839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4f8ce-62ae-402d-93d2-ae8ac2a28aff admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:29.864134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cc21b-0c18-4bcc-a79a-dd9cc0ba9a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:29.866332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cc21b-0c18-4bcc-a79a-dd9cc0ba9a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:29.866632 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cc21b-0c18-4bcc-a79a-dd9cc0ba9a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:29.866962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cc21b-0c18-4bcc-a79a-dd9cc0ba9a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:29.867837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cc21b-0c18-4bcc-a79a-dd9cc0ba9a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:29.869471 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cc21b-0c18-4bcc-a79a-dd9cc0ba9a4b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:29.880892 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1273/2518] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:29.885297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a47738-55ab-4165-84f8-d818de051713 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:29.896917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a47738-55ab-4165-84f8-d818de051713 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:29.901947 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848f0e8-b62d-4462-b086-008cc460918d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:29.902948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f0e8-b62d-4462-b086-008cc460918d admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:29.903471 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f0e8-b62d-4462-b086-008cc460918d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:29.903879 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f0e8-b62d-4462-b086-008cc460918d admin admin] PATH_INFO: `/v3/projects/fd6853316a884b3bb1b01197c43c2d9e/users/602ecb0aaf794ee4a89ffbad75584a01/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:29.905988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848f0e8-b62d-4462-b086-008cc460918d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6853316a884b3bb1b01197c43c2d9e, user_id=602ecb0aaf794ee4a89ffbad75584a01, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:29.908858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848f0e8-b62d-4462-b086-008cc460918d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:29.920548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1246/2519] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:29 2022] PUT /identity/v3/projects/fd6853316a884b3bb1b01197c43c2d9e/users/602ecb0aaf794ee4a89ffbad75584a01/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:29.931934 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9761b9-5592-4769-97c7-d1d94399aa19 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:29.932066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9761b9-5592-4769-97c7-d1d94399aa19 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:29.932868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9761b9-5592-4769-97c7-d1d94399aa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:29.972111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4a9761b9-5592-4769-97c7-d1d94399aa19 None None] MFA Rules not processed for user `602ecb0aaf794ee4a89ffbad75584a01`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:29.973485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4a9761b9-5592-4769-97c7-d1d94399aa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:30.065203 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1274/2520] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:30.079094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848f0e8-b62d-4462-b086-008cc460918d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:30.081034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2848f0e8-b62d-4462-b086-008cc460918d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:30.144668 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848f0e8-b62d-4462-b086-008cc460918d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:30.149483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384a66c-868e-468d-b6f6-0ac7a338fc79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:30.150149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a66c-868e-468d-b6f6-0ac7a338fc79 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:30.150434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a66c-868e-468d-b6f6-0ac7a338fc79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:30.150713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384a66c-868e-468d-b6f6-0ac7a338fc79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:30.162863 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384a66c-868e-468d-b6f6-0ac7a338fc79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:30.164574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384a66c-868e-468d-b6f6-0ac7a338fc79 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:30.176549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1247/2521] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:30.708638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2715c1-03ea-410e-bc96-cf6336eae3d2 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:30.709511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2715c1-03ea-410e-bc96-cf6336eae3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:30.710027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2715c1-03ea-410e-bc96-cf6336eae3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:30.733943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-df2715c1-03ea-410e-bc96-cf6336eae3d2 None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:30.736227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-df2715c1-03ea-410e-bc96-cf6336eae3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:30.753191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:30.807549 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1275/2522] 10.222.0.64 () {58 vars in 1009 bytes} [Mon Jun 27 09:42:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:30.820217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384a66c-868e-468d-b6f6-0ac7a338fc79 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:30.821637 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4384a66c-868e-468d-b6f6-0ac7a338fc79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:30.898277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384a66c-868e-468d-b6f6-0ac7a338fc79 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:30.905445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373fd18-0189-4b63-8cdf-ee3466045b6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:30.906111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373fd18-0189-4b63-8cdf-ee3466045b6c service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:30.906456 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373fd18-0189-4b63-8cdf-ee3466045b6c service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:30.906734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373fd18-0189-4b63-8cdf-ee3466045b6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:30.915809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373fd18-0189-4b63-8cdf-ee3466045b6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:30.917537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373fd18-0189-4b63-8cdf-ee3466045b6c service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:30.927457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1248/2523] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:42:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:42:31.448565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2715c1-03ea-410e-bc96-cf6336eae3d2 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:31.620147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2715c1-03ea-410e-bc96-cf6336eae3d2 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:31.630429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:31.631327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:31.631806 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:31.632211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] PATH_INFO: `/v3/users/eaa48e567e724ad884873c39d25c33e2` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:31.633458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa48e567e724ad884873c39d25c33e2)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:31.635991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:31.896682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa48e567e724ad884873c39d25c33e2'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:31.977935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa48e567e724ad884873c39d25c33e2'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:31.978572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa48e567e724ad884873c39d25c33e2'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:31.986747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa48e567e724ad884873c39d25c33e2'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:31.987454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa48e567e724ad884873c39d25c33e2'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:32.037941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1276/2524] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:31 2022] DELETE /identity/v3/users/eaa48e567e724ad884873c39d25c33e2 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:32.042383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff5b1a-527e-4797-9944-47cdd977ca4a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:32.043093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff5b1a-527e-4797-9944-47cdd977ca4a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:32.043277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff5b1a-527e-4797-9944-47cdd977ca4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:32.064804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-c7ff5b1a-527e-4797-9944-47cdd977ca4a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:32.066306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c7ff5b1a-527e-4797-9944-47cdd977ca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:32.106857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:32.162040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1249/2525] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:42:32.180236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:32.180236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:32.243487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:32.265524 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc7c0f-88b1-4565-8cbd-6496332d00ea admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:32.273689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad116c62-6f35-4f00-865e-8c2be249f976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:32.274363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116c62-6f35-4f00-865e-8c2be249f976 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:32.274645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116c62-6f35-4f00-865e-8c2be249f976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:32.274981 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad116c62-6f35-4f00-865e-8c2be249f976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:32.293976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad116c62-6f35-4f00-865e-8c2be249f976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:32.295595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad116c62-6f35-4f00-865e-8c2be249f976 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:32.308231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1277/2526] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:32.865392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff5b1a-527e-4797-9944-47cdd977ca4a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:32.867186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c7ff5b1a-527e-4797-9944-47cdd977ca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:32.882927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff5b1a-527e-4797-9944-47cdd977ca4a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:32.899583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:32.901360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:32.901750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:32.902099 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] PATH_INFO: `/v3/projects/b0f8a4afc3754ac2a821fca7c9588c57` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:32.903581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f8a4afc3754ac2a821fca7c9588c57)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:32.906290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:33.060437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f8a4afc3754ac2a821fca7c9588c57', 'request_id': 'req-4d634993-cbfb-421a-8b0e-f348b0f9c60f'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.066798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f8a4afc3754ac2a821fca7c9588c57', 'request_id': 'req-4d634993-cbfb-421a-8b0e-f348b0f9c60f'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.074120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f8a4afc3754ac2a821fca7c9588c57', 'request_id': 'req-4d634993-cbfb-421a-8b0e-f348b0f9c60f'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.172616 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] The token cache is being invalidate because project b0f8a4afc3754ac2a821fca7c9588c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:33.172907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.173556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.175176 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1250/2527] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:32 2022] DELETE /identity/v3/projects/b0f8a4afc3754ac2a821fca7c9588c57 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:33.179271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad116c62-6f35-4f00-865e-8c2be249f976 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:33.189709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad116c62-6f35-4f00-865e-8c2be249f976 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:33.245503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:33.246386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:33.246671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:33.246953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] PATH_INFO: `/v3/users/602ecb0aaf794ee4a89ffbad75584a01` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:33.248180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=602ecb0aaf794ee4a89ffbad75584a01)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:33.250161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:33.442235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '602ecb0aaf794ee4a89ffbad75584a01'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.514889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '602ecb0aaf794ee4a89ffbad75584a01'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.515600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '602ecb0aaf794ee4a89ffbad75584a01'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.533161 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '602ecb0aaf794ee4a89ffbad75584a01'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.534044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-9f55abff-26c4-4597-8e2c-d0de33acff9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '602ecb0aaf794ee4a89ffbad75584a01'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:33.573226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1278/2528] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:33 2022] DELETE /identity/v3/users/602ecb0aaf794ee4a89ffbad75584a01 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:34.190882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:34.192420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:34.205812 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d634993-cbfb-421a-8b0e-f348b0f9c60f admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:34.265173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:34.266039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:34.266319 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:34.266593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] PATH_INFO: `/v3/projects/fd6853316a884b3bb1b01197c43c2d9e` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:34.267842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd6853316a884b3bb1b01197c43c2d9e)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:34.269762 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:34.373828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd6853316a884b3bb1b01197c43c2d9e', 'request_id': 'req-7447c810-7368-43db-a783-97e30995aa2d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:34.380160 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:34.381648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd6853316a884b3bb1b01197c43c2d9e', 'request_id': 'req-7447c810-7368-43db-a783-97e30995aa2d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:34.388956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd6853316a884b3bb1b01197c43c2d9e', 'request_id': 'req-7447c810-7368-43db-a783-97e30995aa2d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:34.467648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] The token cache is being invalidate because project fd6853316a884b3bb1b01197c43c2d9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:34.467949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:34.468635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-7447c810-7368-43db-a783-97e30995aa2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:34.470723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1251/2529] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:34 2022] DELETE /identity/v3/projects/fd6853316a884b3bb1b01197c43c2d9e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:34.475420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e820fa-c2de-4253-8ebd-3685ebff2cd2 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:34.475913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e820fa-c2de-4253-8ebd-3685ebff2cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:34.476007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e820fa-c2de-4253-8ebd-3685ebff2cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:34.507286 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c5e820fa-c2de-4253-8ebd-3685ebff2cd2 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:34.508698 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c5e820fa-c2de-4253-8ebd-3685ebff2cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:34.585191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1279/2530] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:34.592894 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e0687-69cc-4f50-900b-eb1de87de6fb None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:34.593188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e0687-69cc-4f50-900b-eb1de87de6fb None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:34.593470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e0687-69cc-4f50-900b-eb1de87de6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:34.612113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-382e0687-69cc-4f50-900b-eb1de87de6fb None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:34.613314 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-382e0687-69cc-4f50-900b-eb1de87de6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:34.622174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1252/2531] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:34.626275 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e820fa-c2de-4253-8ebd-3685ebff2cd2 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:34.632399 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:34.636216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e820fa-c2de-4253-8ebd-3685ebff2cd2 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:34.640863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1dc5a-5840-44be-96d7-f3793052e959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:34.641508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1dc5a-5840-44be-96d7-f3793052e959 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:34.641793 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1dc5a-5840-44be-96d7-f3793052e959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:34.642743 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1dc5a-5840-44be-96d7-f3793052e959 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:34.643057 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1dc5a-5840-44be-96d7-f3793052e959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:42:34.643372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1dc5a-5840-44be-96d7-f3793052e959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:34.645022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1dc5a-5840-44be-96d7-f3793052e959 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:34.657225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1280/2532] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:42:34.662482 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e0687-69cc-4f50-900b-eb1de87de6fb None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:34.676817 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e0687-69cc-4f50-900b-eb1de87de6fb None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:34.681686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121466c6-addc-4803-9544-a613353e5b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:34.682905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121466c6-addc-4803-9544-a613353e5b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:34.683182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121466c6-addc-4803-9544-a613353e5b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:34.683452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121466c6-addc-4803-9544-a613353e5b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:34.684249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121466c6-addc-4803-9544-a613353e5b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:34.686209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121466c6-addc-4803-9544-a613353e5b67 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:34.777355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1253/2533] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:34 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:42:34.778782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1dc5a-5840-44be-96d7-f3793052e959 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:34.789413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1dc5a-5840-44be-96d7-f3793052e959 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:34.882484 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f09b0b-c9b5-46ab-ac58-c6df641f7e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:34.883236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f09b0b-c9b5-46ab-ac58-c6df641f7e3f admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:34.883522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f09b0b-c9b5-46ab-ac58-c6df641f7e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:34.883825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f09b0b-c9b5-46ab-ac58-c6df641f7e3f admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:34.884603 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f09b0b-c9b5-46ab-ac58-c6df641f7e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:34.887049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f09b0b-c9b5-46ab-ac58-c6df641f7e3f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:34.984842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1281/2534] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:34 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:34.989072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121466c6-addc-4803-9544-a613353e5b67 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:34.999847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121466c6-addc-4803-9544-a613353e5b67 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:35.005101 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b551f05-4ec4-46ba-a05d-7dd7c7dced05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:35.006086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b551f05-4ec4-46ba-a05d-7dd7c7dced05 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:35.006655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b551f05-4ec4-46ba-a05d-7dd7c7dced05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:35.007156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b551f05-4ec4-46ba-a05d-7dd7c7dced05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:35.008262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b551f05-4ec4-46ba-a05d-7dd7c7dced05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:35.010338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b551f05-4ec4-46ba-a05d-7dd7c7dced05 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:35.024843 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1254/2535] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:42:35.029333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f09b0b-c9b5-46ab-ac58-c6df641f7e3f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:35.043670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f09b0b-c9b5-46ab-ac58-c6df641f7e3f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:35.048890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef638d-b3fc-4cd3-a8ca-b09031c61f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:35.049784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef638d-b3fc-4cd3-a8ca-b09031c61f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:35.050064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef638d-b3fc-4cd3-a8ca-b09031c61f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:35.050336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef638d-b3fc-4cd3-a8ca-b09031c61f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:35.051064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef638d-b3fc-4cd3-a8ca-b09031c61f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:35.052922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef638d-b3fc-4cd3-a8ca-b09031c61f08 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:35.064062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1282/2536] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:42:35.068191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b551f05-4ec4-46ba-a05d-7dd7c7dced05 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:35.080008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b551f05-4ec4-46ba-a05d-7dd7c7dced05 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:35.086153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfe921-f521-4c31-a737-b1bd249512f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:35.087308 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfe921-f521-4c31-a737-b1bd249512f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:35.087629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfe921-f521-4c31-a737-b1bd249512f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:35.087907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfe921-f521-4c31-a737-b1bd249512f6 admin admin] PATH_INFO: `/v3/projects/cc6a4e3ee95e45479e85087d15914aee/users/6e0dcdee11da4eaabb8d2e0ed490286f/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:35.106052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfe921-f521-4c31-a737-b1bd249512f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6a4e3ee95e45479e85087d15914aee, user_id=6e0dcdee11da4eaabb8d2e0ed490286f, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:35.108765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfe921-f521-4c31-a737-b1bd249512f6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:35.212386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1255/2537] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:35 2022] PUT /identity/v3/projects/cc6a4e3ee95e45479e85087d15914aee/users/6e0dcdee11da4eaabb8d2e0ed490286f/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:35.223266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b050-d88c-442a-b732-b5dcb131957b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:35.223574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b050-d88c-442a-b732-b5dcb131957b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:35.223858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b050-d88c-442a-b732-b5dcb131957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:35.255952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-3332b050-d88c-442a-b732-b5dcb131957b None None] MFA Rules not processed for user `6e0dcdee11da4eaabb8d2e0ed490286f`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:35.257381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3332b050-d88c-442a-b732-b5dcb131957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:35.334385 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1283/2538] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:35.339475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfe921-f521-4c31-a737-b1bd249512f6 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:35.406327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfe921-f521-4c31-a737-b1bd249512f6 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:35.411129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93761d8-f12d-495e-aa74-a134de11f796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:35.411919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93761d8-f12d-495e-aa74-a134de11f796 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:35.412265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93761d8-f12d-495e-aa74-a134de11f796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:35.412605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93761d8-f12d-495e-aa74-a134de11f796 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:35.413376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93761d8-f12d-495e-aa74-a134de11f796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:35.415446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93761d8-f12d-495e-aa74-a134de11f796 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:35.505960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1256/2539] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:35 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:35.510067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b050-d88c-442a-b732-b5dcb131957b None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:35.522129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b050-d88c-442a-b732-b5dcb131957b None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:35.584635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29fb23-a42a-43c5-98e1-c9d678eff277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:35.585533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29fb23-a42a-43c5-98e1-c9d678eff277 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:35.594623 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29fb23-a42a-43c5-98e1-c9d678eff277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:35.595066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e29fb23-a42a-43c5-98e1-c9d678eff277 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:35.595850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29fb23-a42a-43c5-98e1-c9d678eff277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:35.597832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e29fb23-a42a-43c5-98e1-c9d678eff277 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:35.605302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:35.714965 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1284/2540] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:35 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:35.719506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93761d8-f12d-495e-aa74-a134de11f796 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:35.729664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93761d8-f12d-495e-aa74-a134de11f796 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:35.734109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db9b2f-f013-4e9e-80ad-09683dee80c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:35.739026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db9b2f-f013-4e9e-80ad-09683dee80c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:35.739026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db9b2f-f013-4e9e-80ad-09683dee80c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:35.739026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db9b2f-f013-4e9e-80ad-09683dee80c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:35.739026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db9b2f-f013-4e9e-80ad-09683dee80c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:35.739026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db9b2f-f013-4e9e-80ad-09683dee80c9 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:35.748927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1257/2541] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:35.753723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29fb23-a42a-43c5-98e1-c9d678eff277 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:35.768394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e29fb23-a42a-43c5-98e1-c9d678eff277 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:35.775241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9bd24-2f84-4429-838e-b66bb0a56103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:35.776532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9bd24-2f84-4429-838e-b66bb0a56103 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:35.776996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9bd24-2f84-4429-838e-b66bb0a56103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:35.777533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9bd24-2f84-4429-838e-b66bb0a56103 admin admin] PATH_INFO: `/v3/projects/d4de5dae42a44372b1cb387b76810d90/users/08a1b5deedf748c88089bd4e79d079cb/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:35.796545 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9bd24-2f84-4429-838e-b66bb0a56103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4de5dae42a44372b1cb387b76810d90, user_id=08a1b5deedf748c88089bd4e79d079cb, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:35.799519 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9bd24-2f84-4429-838e-b66bb0a56103 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:35.864627 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1285/2542] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:35 2022] PUT /identity/v3/projects/d4de5dae42a44372b1cb387b76810d90/users/08a1b5deedf748c88089bd4e79d079cb/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:35.868405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db9b2f-f013-4e9e-80ad-09683dee80c9 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:35.886500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db9b2f-f013-4e9e-80ad-09683dee80c9 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:35.961260 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:35.976213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e41ea9-0ad4-40a0-8234-24e9732308d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:35.977332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e41ea9-0ad4-40a0-8234-24e9732308d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:35.977773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e41ea9-0ad4-40a0-8234-24e9732308d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:35.978250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e41ea9-0ad4-40a0-8234-24e9732308d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:35.979698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e41ea9-0ad4-40a0-8234-24e9732308d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:35.981695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e41ea9-0ad4-40a0-8234-24e9732308d0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:36.002651 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1258/2543] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:36.006613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9bd24-2f84-4429-838e-b66bb0a56103 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:36.017584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9bd24-2f84-4429-838e-b66bb0a56103 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:36.022239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e80b40-8912-4131-8a9b-983ae52584e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:36.023116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e80b40-8912-4131-8a9b-983ae52584e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:36.023395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e80b40-8912-4131-8a9b-983ae52584e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:36.023705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e80b40-8912-4131-8a9b-983ae52584e6 admin admin] PATH_INFO: `/v3/projects/d4de5dae42a44372b1cb387b76810d90/users/08a1b5deedf748c88089bd4e79d079cb/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:36.025661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e80b40-8912-4131-8a9b-983ae52584e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4de5dae42a44372b1cb387b76810d90, user_id=08a1b5deedf748c88089bd4e79d079cb, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:36.028586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e80b40-8912-4131-8a9b-983ae52584e6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:36.039582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1286/2544] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:36 2022] PUT /identity/v3/projects/d4de5dae42a44372b1cb387b76810d90/users/08a1b5deedf748c88089bd4e79d079cb/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:36.051729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5f648-fe35-40b8-9397-a8a43e0e89cc None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:36.052020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5f648-fe35-40b8-9397-a8a43e0e89cc None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:36.052300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5f648-fe35-40b8-9397-a8a43e0e89cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:36.079842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7ec5f648-fe35-40b8-9397-a8a43e0e89cc None None] MFA Rules not processed for user `08a1b5deedf748c88089bd4e79d079cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:36.081297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7ec5f648-fe35-40b8-9397-a8a43e0e89cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:36.154790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1259/2545] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:36.166379 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e80b40-8912-4131-8a9b-983ae52584e6 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:36.168911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-67e80b40-8912-4131-8a9b-983ae52584e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:36.225578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e80b40-8912-4131-8a9b-983ae52584e6 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:36.229861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c66d1-601b-480b-8e7c-bbb1e37610ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:36.230501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c66d1-601b-480b-8e7c-bbb1e37610ca service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:36.230787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c66d1-601b-480b-8e7c-bbb1e37610ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:36.231065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c66d1-601b-480b-8e7c-bbb1e37610ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:36.240662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c66d1-601b-480b-8e7c-bbb1e37610ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:36.242101 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c66d1-601b-480b-8e7c-bbb1e37610ca service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:36.253463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1287/2546] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:36.420759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5f648-fe35-40b8-9397-a8a43e0e89cc None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:36.431856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5f648-fe35-40b8-9397-a8a43e0e89cc None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:36.435906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73171bf-551b-40dd-903c-435f183ba239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:36.436691 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73171bf-551b-40dd-903c-435f183ba239 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:36.437060 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73171bf-551b-40dd-903c-435f183ba239 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:36.437614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73171bf-551b-40dd-903c-435f183ba239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:36.518153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73171bf-551b-40dd-903c-435f183ba239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:36.519988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73171bf-551b-40dd-903c-435f183ba239 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:36.544886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1260/2547] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:38.156201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c66d1-601b-480b-8e7c-bbb1e37610ca service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:38.264676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c66d1-601b-480b-8e7c-bbb1e37610ca service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:38.274933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:38.276587 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:38.276942 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:38.277137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] PATH_INFO: `/v3/users/6e0dcdee11da4eaabb8d2e0ed490286f` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:38.278674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0dcdee11da4eaabb8d2e0ed490286f)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:38.281505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:38.540449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0dcdee11da4eaabb8d2e0ed490286f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:38.629089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0dcdee11da4eaabb8d2e0ed490286f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:38.629744 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0dcdee11da4eaabb8d2e0ed490286f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:38.632343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:38.640133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0dcdee11da4eaabb8d2e0ed490286f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:38.640963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0dcdee11da4eaabb8d2e0ed490286f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:38.659570 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1288/2548] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:38 2022] DELETE /identity/v3/users/6e0dcdee11da4eaabb8d2e0ed490286f => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:38.664185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d60b-2355-4cc7-8b25-9c6253786707 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:38.664696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d60b-2355-4cc7-8b25-9c6253786707 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:38.664801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d60b-2355-4cc7-8b25-9c6253786707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:38.682650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ae80d60b-2355-4cc7-8b25-9c6253786707 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:38.683895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ae80d60b-2355-4cc7-8b25-9c6253786707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:38.742869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1261/2549] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:38.752539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:38.754028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:38.817433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6513a5a-04c3-441e-9cb6-740867577932 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:38.821331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb02fb-a31d-47a8-8a24-aa9aa945b2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:38.821994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb02fb-a31d-47a8-8a24-aa9aa945b2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:38.822366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb02fb-a31d-47a8-8a24-aa9aa945b2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:38.822646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb02fb-a31d-47a8-8a24-aa9aa945b2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:38.834908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb02fb-a31d-47a8-8a24-aa9aa945b2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:38.836613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb02fb-a31d-47a8-8a24-aa9aa945b2c9 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:38.852024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1289/2550] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:39.316309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80d60b-2355-4cc7-8b25-9c6253786707 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:39.317873 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ae80d60b-2355-4cc7-8b25-9c6253786707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:39.335717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80d60b-2355-4cc7-8b25-9c6253786707 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:39.341580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:39.342446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:39.342783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:39.342993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] PATH_INFO: `/v3/projects/cc6a4e3ee95e45479e85087d15914aee` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:39.344940 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6a4e3ee95e45479e85087d15914aee)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:39.346343 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:39.357441 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:39.453575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6a4e3ee95e45479e85087d15914aee', 'request_id': 'req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:39.458819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6a4e3ee95e45479e85087d15914aee', 'request_id': 'req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:39.465139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6a4e3ee95e45479e85087d15914aee', 'request_id': 'req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:39.578046 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] The token cache is being invalidate because project cc6a4e3ee95e45479e85087d15914aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:39.578458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:39.579187 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:39.580854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1262/2551] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:39 2022] DELETE /identity/v3/projects/cc6a4e3ee95e45479e85087d15914aee => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:39.585119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb02fb-a31d-47a8-8a24-aa9aa945b2c9 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:39.607588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb02fb-a31d-47a8-8a24-aa9aa945b2c9 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:39.678492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:39.678492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:39.678492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:39.678492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] PATH_INFO: `/v3/users/08a1b5deedf748c88089bd4e79d079cb` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:39.678989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a1b5deedf748c88089bd4e79d079cb)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:39.680939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:39.932888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a1b5deedf748c88089bd4e79d079cb'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.021575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a1b5deedf748c88089bd4e79d079cb'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.025631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a1b5deedf748c88089bd4e79d079cb'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.049117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a1b5deedf748c88089bd4e79d079cb'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.050030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dfcd7c1d-bded-4fa2-8b7f-49d34170d2cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a1b5deedf748c88089bd4e79d079cb'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.097907 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1290/2552] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:39 2022] DELETE /identity/v3/users/08a1b5deedf748c88089bd4e79d079cb => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:40.422115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:40.577313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:40.579013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:40.603012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6820f28-b7b9-4de8-9013-a7a4e0740a7d admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:40.660734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:40.661635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:40.661926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:40.662203 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] PATH_INFO: `/v3/projects/d4de5dae42a44372b1cb387b76810d90` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:40.664346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4de5dae42a44372b1cb387b76810d90)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:40.673283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:40.818259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4de5dae42a44372b1cb387b76810d90', 'request_id': 'req-c7886029-97df-423d-a6c0-56d23c7b8b99'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.824614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4de5dae42a44372b1cb387b76810d90', 'request_id': 'req-c7886029-97df-423d-a6c0-56d23c7b8b99'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.832457 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4de5dae42a44372b1cb387b76810d90', 'request_id': 'req-c7886029-97df-423d-a6c0-56d23c7b8b99'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.924242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] The token cache is being invalidate because project d4de5dae42a44372b1cb387b76810d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:40.924626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.925333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c7886029-97df-423d-a6c0-56d23c7b8b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:40.930548 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1263/2553] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:40 2022] DELETE /identity/v3/projects/d4de5dae42a44372b1cb387b76810d90 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:40.932277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b760ea-a667-41e4-ac14-ef64b8b06c55 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:40.932816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b760ea-a667-41e4-ac14-ef64b8b06c55 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:40.932958 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b760ea-a667-41e4-ac14-ef64b8b06c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:40.952116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-b6b760ea-a667-41e4-ac14-ef64b8b06c55 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:40.953323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b6b760ea-a667-41e4-ac14-ef64b8b06c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:41.012404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1291/2554] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:41.020236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d71253-d2f0-4677-ad09-7e76bdf958b3 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:41.020524 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d71253-d2f0-4677-ad09-7e76bdf958b3 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:41.020807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d71253-d2f0-4677-ad09-7e76bdf958b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:41.039928 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-d7d71253-d2f0-4677-ad09-7e76bdf958b3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:41.041074 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d7d71253-d2f0-4677-ad09-7e76bdf958b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:41.050542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1264/2555] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:41.054634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b760ea-a667-41e4-ac14-ef64b8b06c55 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:41.064833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b760ea-a667-41e4-ac14-ef64b8b06c55 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:41.069745 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4388b-5006-4351-9967-05a0287c126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:41.070414 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4388b-5006-4351-9967-05a0287c126f admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:41.070677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4388b-5006-4351-9967-05a0287c126f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:41.070982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4388b-5006-4351-9967-05a0287c126f admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:41.071742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4388b-5006-4351-9967-05a0287c126f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:42:41.072053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4388b-5006-4351-9967-05a0287c126f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:41.073826 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4388b-5006-4351-9967-05a0287c126f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:41.085538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1292/2556] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:42:41.091591 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d71253-d2f0-4677-ad09-7e76bdf958b3 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:41.101230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d71253-d2f0-4677-ad09-7e76bdf958b3 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:41.106756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ade9ba-74de-4cd0-88c9-dee9057b5aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:41.107422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ade9ba-74de-4cd0-88c9-dee9057b5aed admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:41.107750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ade9ba-74de-4cd0-88c9-dee9057b5aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:41.108023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ade9ba-74de-4cd0-88c9-dee9057b5aed admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:41.108710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ade9ba-74de-4cd0-88c9-dee9057b5aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:41.110604 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ade9ba-74de-4cd0-88c9-dee9057b5aed admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:41.253249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:41.263661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1265/2557] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:41 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:41.265506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4388b-5006-4351-9967-05a0287c126f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:41.294755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4388b-5006-4351-9967-05a0287c126f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:41.372812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0274c2-6369-413a-8db2-00fea267530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:41.375815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0274c2-6369-413a-8db2-00fea267530e admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:41.375815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0274c2-6369-413a-8db2-00fea267530e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:41.375815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0274c2-6369-413a-8db2-00fea267530e admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:41.375815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0274c2-6369-413a-8db2-00fea267530e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:41.376879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0274c2-6369-413a-8db2-00fea267530e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:41.464852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1293/2558] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:41 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:41.465667 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ade9ba-74de-4cd0-88c9-dee9057b5aed admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:41.476135 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ade9ba-74de-4cd0-88c9-dee9057b5aed admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:41.480700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01077a-921d-4ca3-9068-7b49e01531bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:41.481357 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01077a-921d-4ca3-9068-7b49e01531bd admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:41.481640 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01077a-921d-4ca3-9068-7b49e01531bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:41.481917 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01077a-921d-4ca3-9068-7b49e01531bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:41.482639 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01077a-921d-4ca3-9068-7b49e01531bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:41.484331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01077a-921d-4ca3-9068-7b49e01531bd admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:41.494547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1266/2559] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:41.501961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0274c2-6369-413a-8db2-00fea267530e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:41.512768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0274c2-6369-413a-8db2-00fea267530e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:41.517312 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fada1b2-249a-432b-9117-c7534f4b98c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:41.518023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada1b2-249a-432b-9117-c7534f4b98c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:41.518302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada1b2-249a-432b-9117-c7534f4b98c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:41.518575 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada1b2-249a-432b-9117-c7534f4b98c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:41.519291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fada1b2-249a-432b-9117-c7534f4b98c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:41.520911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fada1b2-249a-432b-9117-c7534f4b98c0 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:41.533189 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1294/2560] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:41.537326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01077a-921d-4ca3-9068-7b49e01531bd admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:41.547757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01077a-921d-4ca3-9068-7b49e01531bd admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:41.552661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3384d-5521-41ec-bc40-27b1979362f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:41.553583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3384d-5521-41ec-bc40-27b1979362f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:41.553860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3384d-5521-41ec-bc40-27b1979362f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:41.554134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3384d-5521-41ec-bc40-27b1979362f0 admin admin] PATH_INFO: `/v3/projects/24dd3a90af884ec48ce5fb8484d6f2e1/users/2d324f57032b4b44a5157b8d093f9bb3/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:41.568654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3384d-5521-41ec-bc40-27b1979362f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24dd3a90af884ec48ce5fb8484d6f2e1, user_id=2d324f57032b4b44a5157b8d093f9bb3, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:41.571416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3384d-5521-41ec-bc40-27b1979362f0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:41.638647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1267/2561] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:41 2022] PUT /identity/v3/projects/24dd3a90af884ec48ce5fb8484d6f2e1/users/2d324f57032b4b44a5157b8d093f9bb3/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:41.649692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313f45d-b8b0-46d1-86a9-4d2169ee350f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:41.649980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313f45d-b8b0-46d1-86a9-4d2169ee350f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:41.650281 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313f45d-b8b0-46d1-86a9-4d2169ee350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:41.676395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4313f45d-b8b0-46d1-86a9-4d2169ee350f None None] MFA Rules not processed for user `2d324f57032b4b44a5157b8d093f9bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:41.677626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4313f45d-b8b0-46d1-86a9-4d2169ee350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:41.724719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:41.740912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1295/2562] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:41.745831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3384d-5521-41ec-bc40-27b1979362f0 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:41.807383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3384d-5521-41ec-bc40-27b1979362f0 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:41.812124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd22f61-6c16-435f-b7eb-e4816ec55338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:41.812779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd22f61-6c16-435f-b7eb-e4816ec55338 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:41.813061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd22f61-6c16-435f-b7eb-e4816ec55338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:41.813336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd22f61-6c16-435f-b7eb-e4816ec55338 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:41.814078 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd22f61-6c16-435f-b7eb-e4816ec55338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:41.816026 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd22f61-6c16-435f-b7eb-e4816ec55338 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:41.943565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1268/2563] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:41 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:42:41.948267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313f45d-b8b0-46d1-86a9-4d2169ee350f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:41.959270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313f45d-b8b0-46d1-86a9-4d2169ee350f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:42.016290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b43f0-452a-4c06-ab3b-9427f82d1a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:42.016984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b43f0-452a-4c06-ab3b-9427f82d1a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:42.017267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b43f0-452a-4c06-ab3b-9427f82d1a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:42.017542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b43f0-452a-4c06-ab3b-9427f82d1a09 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:42.018261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b43f0-452a-4c06-ab3b-9427f82d1a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:42.020214 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b43f0-452a-4c06-ab3b-9427f82d1a09 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:42.098196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1296/2564] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:41 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:42.100140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd22f61-6c16-435f-b7eb-e4816ec55338 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:42.110989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd22f61-6c16-435f-b7eb-e4816ec55338 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:42.115700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c18f01-3002-4f26-8926-b40f1275b176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:42.116349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c18f01-3002-4f26-8926-b40f1275b176 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:42.116630 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c18f01-3002-4f26-8926-b40f1275b176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:42.116904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c18f01-3002-4f26-8926-b40f1275b176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:42.117689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c18f01-3002-4f26-8926-b40f1275b176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:42.119395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c18f01-3002-4f26-8926-b40f1275b176 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:42.130378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1269/2565] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:42:42.136328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b43f0-452a-4c06-ab3b-9427f82d1a09 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:42.146437 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b43f0-452a-4c06-ab3b-9427f82d1a09 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:42.151010 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d192c8d-e976-4493-bfb6-0d515cf08d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:42.151926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d192c8d-e976-4493-bfb6-0d515cf08d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:42.152212 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d192c8d-e976-4493-bfb6-0d515cf08d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:42.152489 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d192c8d-e976-4493-bfb6-0d515cf08d0d admin admin] PATH_INFO: `/v3/projects/f8c6078ffca447f787ecb13c61fe7134/users/0fa25bb3cdb949829f57f38f52b213c7/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:42.167347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d192c8d-e976-4493-bfb6-0d515cf08d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c6078ffca447f787ecb13c61fe7134, user_id=0fa25bb3cdb949829f57f38f52b213c7, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:42.170315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d192c8d-e976-4493-bfb6-0d515cf08d0d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:42.263047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1297/2566] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:42 2022] PUT /identity/v3/projects/f8c6078ffca447f787ecb13c61fe7134/users/0fa25bb3cdb949829f57f38f52b213c7/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:42.267183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c18f01-3002-4f26-8926-b40f1275b176 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:42.277398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c18f01-3002-4f26-8926-b40f1275b176 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:42.333155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d21d0-2d7e-41b5-94a7-cf0bf25eeec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:42.333811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d21d0-2d7e-41b5-94a7-cf0bf25eeec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:42.334090 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d21d0-2d7e-41b5-94a7-cf0bf25eeec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:42.334365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d21d0-2d7e-41b5-94a7-cf0bf25eeec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:42.335075 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d21d0-2d7e-41b5-94a7-cf0bf25eeec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:42.336693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d21d0-2d7e-41b5-94a7-cf0bf25eeec6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:42.346643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1270/2567] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:42.350815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d192c8d-e976-4493-bfb6-0d515cf08d0d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:42.362264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d192c8d-e976-4493-bfb6-0d515cf08d0d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:42.367058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be12b-e623-48ed-91bf-1c95767cab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:42.367988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be12b-e623-48ed-91bf-1c95767cab44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:42.368271 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be12b-e623-48ed-91bf-1c95767cab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:42.368546 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be12b-e623-48ed-91bf-1c95767cab44 admin admin] PATH_INFO: `/v3/projects/f8c6078ffca447f787ecb13c61fe7134/users/0fa25bb3cdb949829f57f38f52b213c7/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:42.370613 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be12b-e623-48ed-91bf-1c95767cab44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c6078ffca447f787ecb13c61fe7134, user_id=0fa25bb3cdb949829f57f38f52b213c7, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:42.373423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be12b-e623-48ed-91bf-1c95767cab44 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:42.384209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1298/2568] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:42 2022] PUT /identity/v3/projects/f8c6078ffca447f787ecb13c61fe7134/users/0fa25bb3cdb949829f57f38f52b213c7/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:42.394407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445047da-6570-4c11-a5ed-e495c548ce71 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:42.394702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445047da-6570-4c11-a5ed-e495c548ce71 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:42.394992 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445047da-6570-4c11-a5ed-e495c548ce71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:42.425529 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-445047da-6570-4c11-a5ed-e495c548ce71 None None] MFA Rules not processed for user `0fa25bb3cdb949829f57f38f52b213c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:42.426731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-445047da-6570-4c11-a5ed-e495c548ce71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:42.451455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:42.500506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1271/2569] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:42.512184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be12b-e623-48ed-91bf-1c95767cab44 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:42.514087 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-da2be12b-e623-48ed-91bf-1c95767cab44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:42.579046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be12b-e623-48ed-91bf-1c95767cab44 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:42.582697 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91915c2-2d2e-40a6-b894-83f37c7a3d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:42.583344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91915c2-2d2e-40a6-b894-83f37c7a3d9b service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:42.583657 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91915c2-2d2e-40a6-b894-83f37c7a3d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:42.583936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91915c2-2d2e-40a6-b894-83f37c7a3d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:42.644953 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91915c2-2d2e-40a6-b894-83f37c7a3d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:42.646397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91915c2-2d2e-40a6-b894-83f37c7a3d9b service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:42.649998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:42.656473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1299/2570] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:43.561776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445047da-6570-4c11-a5ed-e495c548ce71 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:43.563279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-445047da-6570-4c11-a5ed-e495c548ce71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:43.654969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445047da-6570-4c11-a5ed-e495c548ce71 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:43.660856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745f9ac-f931-4e8c-8d23-eccfbb8f9e2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:43.661904 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f9ac-f931-4e8c-8d23-eccfbb8f9e2e service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:43.662261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f9ac-f931-4e8c-8d23-eccfbb8f9e2e service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:43.662737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f9ac-f931-4e8c-8d23-eccfbb8f9e2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:43.665240 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a745f9ac-f931-4e8c-8d23-eccfbb8f9e2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:43.770490 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745f9ac-f931-4e8c-8d23-eccfbb8f9e2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:43.773042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745f9ac-f931-4e8c-8d23-eccfbb8f9e2e service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:43.784374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1272/2571] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:44.360001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91915c2-2d2e-40a6-b894-83f37c7a3d9b service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:44.361503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b91915c2-2d2e-40a6-b894-83f37c7a3d9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:44.378223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91915c2-2d2e-40a6-b894-83f37c7a3d9b service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:44.383835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ee467-9e54-4e9c-b614-e8dbeaced9de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:44.384483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ee467-9e54-4e9c-b614-e8dbeaced9de service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:44.384768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ee467-9e54-4e9c-b614-e8dbeaced9de service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:44.385084 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ee467-9e54-4e9c-b614-e8dbeaced9de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:44.386911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-799ee467-9e54-4e9c-b614-e8dbeaced9de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:44.399384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ee467-9e54-4e9c-b614-e8dbeaced9de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:44.401056 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ee467-9e54-4e9c-b614-e8dbeaced9de service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:44.411754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1300/2572] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:45.945513 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6188f-a160-4649-9852-35af5b58fe68 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:45.946023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6188f-a160-4649-9852-35af5b58fe68 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:45.946127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6188f-a160-4649-9852-35af5b58fe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:45.968870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-fdd6188f-a160-4649-9852-35af5b58fe68 None None] MFA Rules not processed for user `2720d45854ab4a61b3b22dd43a523d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:45.970407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fdd6188f-a160-4649-9852-35af5b58fe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:46.030515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1273/2573] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:46.986944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c044-40f6-44d6-8895-2516cc1a629a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:46.987411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c044-40f6-44d6-8895-2516cc1a629a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:46.987558 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c044-40f6-44d6-8895-2516cc1a629a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:47.005539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-cf64c044-40f6-44d6-8895-2516cc1a629a None None] MFA Rules not processed for user `8731ac7e5a7941f1b2ad04bddc966db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:47.006735 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-cf64c044-40f6-44d6-8895-2516cc1a629a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:47.069442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1301/2574] 10.222.0.64 () {58 vars in 967 bytes} [Mon Jun 27 09:42:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:47.081853 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6188f-a160-4649-9852-35af5b58fe68 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:47.083326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fdd6188f-a160-4649-9852-35af5b58fe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:47.099832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:47.227164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6188f-a160-4649-9852-35af5b58fe68 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:47.232379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802cb6c-83c2-4fff-a815-89413f409f67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70b499a32516486d9a2bed34f7e66b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:47.233177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802cb6c-83c2-4fff-a815-89413f409f67 service swift] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:47.234244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802cb6c-83c2-4fff-a815-89413f409f67 service swift] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:47.234536 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802cb6c-83c2-4fff-a815-89413f409f67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:47.243845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802cb6c-83c2-4fff-a815-89413f409f67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:47.245658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802cb6c-83c2-4fff-a815-89413f409f67 service swift] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:47.255208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1274/2575] 10.222.0.64 () {60 vars in 1384 bytes} [Mon Jun 27 09:42:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 09:42:47.857309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64c044-40f6-44d6-8895-2516cc1a629a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:47.871411 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64c044-40f6-44d6-8895-2516cc1a629a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:47.875750 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd312e-f7ef-457a-9e29-eba89f384851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:47.876473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd312e-f7ef-457a-9e29-eba89f384851 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:47.879650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd312e-f7ef-457a-9e29-eba89f384851 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:47.879650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd312e-f7ef-457a-9e29-eba89f384851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:47.887915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd312e-f7ef-457a-9e29-eba89f384851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:47.889767 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd312e-f7ef-457a-9e29-eba89f384851 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:47.902091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1302/2576] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:53.894402 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802cb6c-83c2-4fff-a815-89413f409f67 service swift] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:53.955652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802cb6c-83c2-4fff-a815-89413f409f67 service swift] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:53.960318 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:53.961225 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:53.961511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:53.961790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] PATH_INFO: `/v3/users/2d324f57032b4b44a5157b8d093f9bb3` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:53.963139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d324f57032b4b44a5157b8d093f9bb3)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:53.965291 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:54.167344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d324f57032b4b44a5157b8d093f9bb3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:54.168351 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d324f57032b4b44a5157b8d093f9bb3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:54.175801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d324f57032b4b44a5157b8d093f9bb3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:54.176563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d324f57032b4b44a5157b8d093f9bb3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:54.189775 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d324f57032b4b44a5157b8d093f9bb3'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:54.250443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1275/2577] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:53 2022] DELETE /identity/v3/users/2d324f57032b4b44a5157b8d093f9bb3 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:54.255062 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7356d8b-e48a-4d8b-a6e1-d44951b330e6 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:54.255633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7356d8b-e48a-4d8b-a6e1-d44951b330e6 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:54.255633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7356d8b-e48a-4d8b-a6e1-d44951b330e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:54.273629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d7356d8b-e48a-4d8b-a6e1-d44951b330e6 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:54.274966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d7356d8b-e48a-4d8b-a6e1-d44951b330e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:54.334299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1303/2578] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:54.344452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:54.345839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:54.359895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:54.403383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626de41-b862-404e-a224-53ee2ff2a5a6 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:54.407305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896193db-3f09-4f4c-8f95-3249ad1b0dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:54.409266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896193db-3f09-4f4c-8f95-3249ad1b0dbb service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:54.409266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896193db-3f09-4f4c-8f95-3249ad1b0dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:54.409266 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896193db-3f09-4f4c-8f95-3249ad1b0dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:54.418801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896193db-3f09-4f4c-8f95-3249ad1b0dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:54.420336 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896193db-3f09-4f4c-8f95-3249ad1b0dbb service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:54.430593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1276/2579] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:42:54.816835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7356d8b-e48a-4d8b-a6e1-d44951b330e6 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:54.821779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d7356d8b-e48a-4d8b-a6e1-d44951b330e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:54.831651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7356d8b-e48a-4d8b-a6e1-d44951b330e6 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:54.836501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:54.837412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:54.837693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:54.837966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] PATH_INFO: `/v3/projects/24dd3a90af884ec48ce5fb8484d6f2e1` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:54.840078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=24dd3a90af884ec48ce5fb8484d6f2e1)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:54.842076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:54.946104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24dd3a90af884ec48ce5fb8484d6f2e1', 'request_id': 'req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.016868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24dd3a90af884ec48ce5fb8484d6f2e1', 'request_id': 'req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.022572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24dd3a90af884ec48ce5fb8484d6f2e1', 'request_id': 'req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.030845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:55.035073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] The token cache is being invalidate because project 24dd3a90af884ec48ce5fb8484d6f2e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:55.035365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.036029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.037565 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1304/2580] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:54 2022] DELETE /identity/v3/projects/24dd3a90af884ec48ce5fb8484d6f2e1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:55.041591 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896193db-3f09-4f4c-8f95-3249ad1b0dbb service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:55.052862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896193db-3f09-4f4c-8f95-3249ad1b0dbb service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:55.109409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:55.110194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:55.110476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:55.110750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] PATH_INFO: `/v3/users/0fa25bb3cdb949829f57f38f52b213c7` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:55.111965 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fa25bb3cdb949829f57f38f52b213c7)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:55.113959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:55.304735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa25bb3cdb949829f57f38f52b213c7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.305384 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa25bb3cdb949829f57f38f52b213c7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.313015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa25bb3cdb949829f57f38f52b213c7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.313749 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa25bb3cdb949829f57f38f52b213c7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.326480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e3a963f4-452d-4341-a8e4-2cb60634eaf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa25bb3cdb949829f57f38f52b213c7'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.411373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1277/2581] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:42:55 2022] DELETE /identity/v3/users/0fa25bb3cdb949829f57f38f52b213c7 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:55.790131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:55.792149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:55.806241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16720bf5-efe7-4225-b7a2-cbeb92edfe9f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:55.863679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:55.864542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:55.864820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:55.865091 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] PATH_INFO: `/v3/projects/f8c6078ffca447f787ecb13c61fe7134` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:55.866267 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c6078ffca447f787ecb13c61fe7134)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:55.868225 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:55.959311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c6078ffca447f787ecb13c61fe7134', 'request_id': 'req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:55.999768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c6078ffca447f787ecb13c61fe7134', 'request_id': 'req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:56.005290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c6078ffca447f787ecb13c61fe7134', 'request_id': 'req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:56.016321 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] The token cache is being invalidate because project f8c6078ffca447f787ecb13c61fe7134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:42:56.016609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:56.017230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-643d6ad0-ab2a-49f6-ab39-e06ae13dd311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:42:56.018751 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1305/2582] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:42:55 2022] DELETE /identity/v3/projects/f8c6078ffca447f787ecb13c61fe7134 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:42:56.024842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d51d6-9590-46ea-b0f0-b0a491601002 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.025155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d51d6-9590-46ea-b0f0-b0a491601002 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.025431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d51d6-9590-46ea-b0f0-b0a491601002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.044526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-da8d51d6-9590-46ea-b0f0-b0a491601002 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:56.045714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-da8d51d6-9590-46ea-b0f0-b0a491601002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:56.075104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:56.113367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1278/2583] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:56.122144 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ede945-0e1d-42f0-b327-3fafedcdadb8 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.122436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ede945-0e1d-42f0-b327-3fafedcdadb8 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.122713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ede945-0e1d-42f0-b327-3fafedcdadb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.141042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-97ede945-0e1d-42f0-b327-3fafedcdadb8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:56.142317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-97ede945-0e1d-42f0-b327-3fafedcdadb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:56.150579 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1306/2584] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:56.155488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d51d6-9590-46ea-b0f0-b0a491601002 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:56.165484 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d51d6-9590-46ea-b0f0-b0a491601002 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:56.170996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953a3f0-975e-42b2-b6e1-e12f5df5a244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:56.171680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953a3f0-975e-42b2-b6e1-e12f5df5a244 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.171969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953a3f0-975e-42b2-b6e1-e12f5df5a244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.172249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953a3f0-975e-42b2-b6e1-e12f5df5a244 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.172975 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953a3f0-975e-42b2-b6e1-e12f5df5a244 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:42:56.173287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953a3f0-975e-42b2-b6e1-e12f5df5a244 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:56.174892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953a3f0-975e-42b2-b6e1-e12f5df5a244 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:56.186642 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1279/2585] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:42:56.190888 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ede945-0e1d-42f0-b327-3fafedcdadb8 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:56.200633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ede945-0e1d-42f0-b327-3fafedcdadb8 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:56.205291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dda368-bd4a-4fe3-93e0-aa7c9e1f6aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:56.205938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dda368-bd4a-4fe3-93e0-aa7c9e1f6aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.206219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dda368-bd4a-4fe3-93e0-aa7c9e1f6aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.206497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dda368-bd4a-4fe3-93e0-aa7c9e1f6aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.207179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dda368-bd4a-4fe3-93e0-aa7c9e1f6aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:56.209102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dda368-bd4a-4fe3-93e0-aa7c9e1f6aa3 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:56.282320 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1307/2586] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:42:56 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:42:56.286605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953a3f0-975e-42b2-b6e1-e12f5df5a244 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:56.297491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953a3f0-975e-42b2-b6e1-e12f5df5a244 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:56.353290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67297599-3eae-4374-a1c6-59a3ae672977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:56.353960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67297599-3eae-4374-a1c6-59a3ae672977 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.354248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67297599-3eae-4374-a1c6-59a3ae672977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.354531 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67297599-3eae-4374-a1c6-59a3ae672977 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.355228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67297599-3eae-4374-a1c6-59a3ae672977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:56.357201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67297599-3eae-4374-a1c6-59a3ae672977 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:56.424177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1280/2587] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:42:56 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:42:56.428260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dda368-bd4a-4fe3-93e0-aa7c9e1f6aa3 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:56.439475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dda368-bd4a-4fe3-93e0-aa7c9e1f6aa3 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:56.444177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8f10-2c78-4860-83f7-d5bde948dffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:56.444832 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8f10-2c78-4860-83f7-d5bde948dffd admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.445118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8f10-2c78-4860-83f7-d5bde948dffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.445466 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8f10-2c78-4860-83f7-d5bde948dffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.446112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef8f10-2c78-4860-83f7-d5bde948dffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:56.448256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef8f10-2c78-4860-83f7-d5bde948dffd admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:56.458853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1308/2588] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:56.462820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67297599-3eae-4374-a1c6-59a3ae672977 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:56.473270 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67297599-3eae-4374-a1c6-59a3ae672977 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:56.477893 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8b6f3-27c0-4642-bd0d-72971685b389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:56.478567 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8b6f3-27c0-4642-bd0d-72971685b389 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.478847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8b6f3-27c0-4642-bd0d-72971685b389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.483485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8b6f3-27c0-4642-bd0d-72971685b389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.483485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede8b6f3-27c0-4642-bd0d-72971685b389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:56.483485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede8b6f3-27c0-4642-bd0d-72971685b389 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:56.491798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1281/2589] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:42:56.496333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8f10-2c78-4860-83f7-d5bde948dffd admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:56.506837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8f10-2c78-4860-83f7-d5bde948dffd admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:56.512171 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2352d3-793c-474e-a704-c3e20041ebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:56.513094 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2352d3-793c-474e-a704-c3e20041ebab admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.513381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2352d3-793c-474e-a704-c3e20041ebab admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.513659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2352d3-793c-474e-a704-c3e20041ebab admin admin] PATH_INFO: `/v3/projects/f1d306a47dd24f938d623383dcab5fbf/users/834823b0e5424a2a851f06eb0e248957/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.529557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2352d3-793c-474e-a704-c3e20041ebab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d306a47dd24f938d623383dcab5fbf, user_id=834823b0e5424a2a851f06eb0e248957, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:56.532492 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2352d3-793c-474e-a704-c3e20041ebab admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:56.611177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1309/2590] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:42:56 2022] PUT /identity/v3/projects/f1d306a47dd24f938d623383dcab5fbf/users/834823b0e5424a2a851f06eb0e248957/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:42:56.674430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8a974-10a0-414b-9f48-a53af7124aeb None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.674720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8a974-10a0-414b-9f48-a53af7124aeb None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.675001 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8a974-10a0-414b-9f48-a53af7124aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.702220 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-10f8a974-10a0-414b-9f48-a53af7124aeb None None] MFA Rules not processed for user `834823b0e5424a2a851f06eb0e248957`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:42:56.703410 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-10f8a974-10a0-414b-9f48-a53af7124aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:56.766250 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1282/2591] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:42:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:42:56.777392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2352d3-793c-474e-a704-c3e20041ebab admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:42:56.779035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1e2352d3-793c-474e-a704-c3e20041ebab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:42:56.809792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:42:56.856370 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2352d3-793c-474e-a704-c3e20041ebab admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:42:56.860069 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8b281-d6cc-4d1f-90c0-672cb1154118 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e33d1473077247cc97dcfd99b3da5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:42:56.860704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8b281-d6cc-4d1f-90c0-672cb1154118 service glance] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:42:56.861008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8b281-d6cc-4d1f-90c0-672cb1154118 service glance] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:42:56.861283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8b281-d6cc-4d1f-90c0-672cb1154118 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:42:56.870308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8b281-d6cc-4d1f-90c0-672cb1154118 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:42:56.871747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8b281-d6cc-4d1f-90c0-672cb1154118 service glance] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:42:56.881216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1310/2592] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:42:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:43:06.763381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:06.902722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=23 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:13.964766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8a974-10a0-414b-9f48-a53af7124aeb None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:14.024990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8a974-10a0-414b-9f48-a53af7124aeb None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:14.029510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:14.030373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:14.030657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:14.030934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] PATH_INFO: `/v3/users/834823b0e5424a2a851f06eb0e248957` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:14.032130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=834823b0e5424a2a851f06eb0e248957)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:14.034047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:14.218454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834823b0e5424a2a851f06eb0e248957'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:14.219041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834823b0e5424a2a851f06eb0e248957'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:14.225926 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834823b0e5424a2a851f06eb0e248957'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:14.226556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834823b0e5424a2a851f06eb0e248957'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:14.238694 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834823b0e5424a2a851f06eb0e248957'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:14.317937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1283/2593] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:43:13 2022] DELETE /identity/v3/users/834823b0e5424a2a851f06eb0e248957 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:14.322434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cb8e0-21b5-45ed-ba92-466923f14a58 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:14.323707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cb8e0-21b5-45ed-ba92-466923f14a58 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:14.323707 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cb8e0-21b5-45ed-ba92-466923f14a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:14.340683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-587cb8e0-21b5-45ed-ba92-466923f14a58 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:14.342048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-587cb8e0-21b5-45ed-ba92-466923f14a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:14.399305 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1311/2594] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:14.408753 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:14.410156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:14.466677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7018e-27c6-46ec-9993-8a0b34ca5055 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:14.470377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af8e95-3c1e-4dd3-895d-f1e2f1ec3c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:14.471004 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af8e95-3c1e-4dd3-895d-f1e2f1ec3c1e service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:14.471267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af8e95-3c1e-4dd3-895d-f1e2f1ec3c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:14.471569 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af8e95-3c1e-4dd3-895d-f1e2f1ec3c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:14.480935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af8e95-3c1e-4dd3-895d-f1e2f1ec3c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:14.482353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af8e95-3c1e-4dd3-895d-f1e2f1ec3c1e service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:14.491704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1284/2595] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:43:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:43:14.915423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cb8e0-21b5-45ed-ba92-466923f14a58 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:14.916904 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-587cb8e0-21b5-45ed-ba92-466923f14a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:14.931023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cb8e0-21b5-45ed-ba92-466923f14a58 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:14.935786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:14.937369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:14.937661 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:14.937938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] PATH_INFO: `/v3/projects/f1d306a47dd24f938d623383dcab5fbf` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:14.939149 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d306a47dd24f938d623383dcab5fbf)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:14.941076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:15.062153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d306a47dd24f938d623383dcab5fbf', 'request_id': 'req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:15.125365 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d306a47dd24f938d623383dcab5fbf', 'request_id': 'req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:15.131133 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d306a47dd24f938d623383dcab5fbf', 'request_id': 'req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:15.142856 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] The token cache is being invalidate because project f1d306a47dd24f938d623383dcab5fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:43:15.144450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:15.144450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:15.145338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1312/2596] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:43:14 2022] DELETE /identity/v3/projects/f1d306a47dd24f938d623383dcab5fbf => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:24.487506 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:25.139882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:35.557154 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af8e95-3c1e-4dd3-895d-f1e2f1ec3c1e service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:35.558585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-59af8e95-3c1e-4dd3-895d-f1e2f1ec3c1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:35.569719 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af8e95-3c1e-4dd3-895d-f1e2f1ec3c1e service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:35.617573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a64d61-7ca7-4bc5-a2bd-94e895ecb495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:35.618243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a64d61-7ca7-4bc5-a2bd-94e895ecb495 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:35.618528 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a64d61-7ca7-4bc5-a2bd-94e895ecb495 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:35.618807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a64d61-7ca7-4bc5-a2bd-94e895ecb495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:35.620624 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b6a64d61-7ca7-4bc5-a2bd-94e895ecb495 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:35.682734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a64d61-7ca7-4bc5-a2bd-94e895ecb495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:35.684222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a64d61-7ca7-4bc5-a2bd-94e895ecb495 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:35.694289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1285/2597] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:43:39.288706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:39.290705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:39.306280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0faf4-e5af-40cd-b0ff-ebb3bab96545 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:39.311939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:39.312824 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:39.313113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:39.313395 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] PATH_INFO: `/v3/users/ecf085030e8e4a65a7140fea2b64a7ed` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:39.314634 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf085030e8e4a65a7140fea2b64a7ed)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:39.316643 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:39.740723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf085030e8e4a65a7140fea2b64a7ed'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:39.813901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf085030e8e4a65a7140fea2b64a7ed'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:39.813901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf085030e8e4a65a7140fea2b64a7ed'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:39.820995 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf085030e8e4a65a7140fea2b64a7ed'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:39.821905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-489247a9-8fcf-442c-a5e6-252f412c9d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf085030e8e4a65a7140fea2b64a7ed'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:39.840097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1313/2598] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:43:39 2022] DELETE /identity/v3/users/ecf085030e8e4a65a7140fea2b64a7ed => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:43:40.116110 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a64d61-7ca7-4bc5-a2bd-94e895ecb495 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:40.134022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a64d61-7ca7-4bc5-a2bd-94e895ecb495 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:40.187726 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:40.192174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:40.192174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:40.192174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] PATH_INFO: `/v3/projects/79203b0660c041f49c5f2629bd5a0bdf` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:40.192174 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79203b0660c041f49c5f2629bd5a0bdf)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:40.192574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:40.339376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79203b0660c041f49c5f2629bd5a0bdf', 'request_id': 'req-b753bc3d-5202-4ff1-bcea-de9404c8875d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:40.341374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:40.345831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79203b0660c041f49c5f2629bd5a0bdf', 'request_id': 'req-b753bc3d-5202-4ff1-bcea-de9404c8875d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:40.352065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79203b0660c041f49c5f2629bd5a0bdf', 'request_id': 'req-b753bc3d-5202-4ff1-bcea-de9404c8875d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:40.399273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] The token cache is being invalidate because project 79203b0660c041f49c5f2629bd5a0bdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:43:40.399589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:40.400265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-b753bc3d-5202-4ff1-bcea-de9404c8875d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:40.401935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1286/2599] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:43:40 2022] DELETE /identity/v3/projects/79203b0660c041f49c5f2629bd5a0bdf => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:40.407616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5b996-c679-4360-bee3-ba1efb98e9ba None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:40.408130 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5b996-c679-4360-bee3-ba1efb98e9ba None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:40.408231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5b996-c679-4360-bee3-ba1efb98e9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:40.426352 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8ec5b996-c679-4360-bee3-ba1efb98e9ba None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:40.427500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8ec5b996-c679-4360-bee3-ba1efb98e9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:40.488286 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1314/2600] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:40.496372 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc21edd-6d0f-4c05-88ed-c0c57b274d1f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:40.496664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc21edd-6d0f-4c05-88ed-c0c57b274d1f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:40.496942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc21edd-6d0f-4c05-88ed-c0c57b274d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:40.515538 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-dfc21edd-6d0f-4c05-88ed-c0c57b274d1f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:40.516768 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-dfc21edd-6d0f-4c05-88ed-c0c57b274d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:40.524686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1287/2601] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:40.528752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5b996-c679-4360-bee3-ba1efb98e9ba None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:40.539180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5b996-c679-4360-bee3-ba1efb98e9ba None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:40.544256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56634735-2584-433c-afc7-5a23ebcb92f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:40.544946 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56634735-2584-433c-afc7-5a23ebcb92f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:40.545235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56634735-2584-433c-afc7-5a23ebcb92f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:40.545533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56634735-2584-433c-afc7-5a23ebcb92f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:40.546258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56634735-2584-433c-afc7-5a23ebcb92f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:43:40.546572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56634735-2584-433c-afc7-5a23ebcb92f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:40.548199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56634735-2584-433c-afc7-5a23ebcb92f9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:40.559538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1315/2602] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:43:40.563553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc21edd-6d0f-4c05-88ed-c0c57b274d1f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:40.572686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc21edd-6d0f-4c05-88ed-c0c57b274d1f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:40.577886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0bc2c-f4cf-43ae-adba-464d2b2ca0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:40.578528 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0bc2c-f4cf-43ae-adba-464d2b2ca0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:40.578813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0bc2c-f4cf-43ae-adba-464d2b2ca0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:40.579097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0bc2c-f4cf-43ae-adba-464d2b2ca0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:40.579829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0bc2c-f4cf-43ae-adba-464d2b2ca0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:40.581712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0bc2c-f4cf-43ae-adba-464d2b2ca0a3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:40.662649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1288/2603] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:43:40 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:43:40.667058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56634735-2584-433c-afc7-5a23ebcb92f9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:40.677601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56634735-2584-433c-afc7-5a23ebcb92f9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:40.734034 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ab0b6-ecd3-4e62-97bb-53377ab93a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:40.735339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ab0b6-ecd3-4e62-97bb-53377ab93a02 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:40.735845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ab0b6-ecd3-4e62-97bb-53377ab93a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:40.736142 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ab0b6-ecd3-4e62-97bb-53377ab93a02 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:40.736835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ab0b6-ecd3-4e62-97bb-53377ab93a02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:40.738855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ab0b6-ecd3-4e62-97bb-53377ab93a02 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:40.745952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:40.837943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1316/2604] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:43:40 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:43:40.839362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0bc2c-f4cf-43ae-adba-464d2b2ca0a3 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:40.850474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0bc2c-f4cf-43ae-adba-464d2b2ca0a3 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:40.855662 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae182a-ba98-41db-8d10-d3a8aa21912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:40.861473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae182a-ba98-41db-8d10-d3a8aa21912c admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:40.861473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae182a-ba98-41db-8d10-d3a8aa21912c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:40.861473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae182a-ba98-41db-8d10-d3a8aa21912c admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:40.861473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae182a-ba98-41db-8d10-d3a8aa21912c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:40.861473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae182a-ba98-41db-8d10-d3a8aa21912c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:40.870892 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1289/2605] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:43:40.875305 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ab0b6-ecd3-4e62-97bb-53377ab93a02 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:40.885992 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ab0b6-ecd3-4e62-97bb-53377ab93a02 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:40.891394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e38c40-934f-4500-8b3b-05c459830468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:40.892082 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e38c40-934f-4500-8b3b-05c459830468 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:40.892371 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e38c40-934f-4500-8b3b-05c459830468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:40.892685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e38c40-934f-4500-8b3b-05c459830468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:40.893413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e38c40-934f-4500-8b3b-05c459830468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:40.895017 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e38c40-934f-4500-8b3b-05c459830468 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:40.905975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1317/2606] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:43:40.910289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae182a-ba98-41db-8d10-d3a8aa21912c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:40.921103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae182a-ba98-41db-8d10-d3a8aa21912c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:40.926064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1ec4d-5cb7-43f3-9c73-e0f40a2a52bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:40.926948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1ec4d-5cb7-43f3-9c73-e0f40a2a52bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:40.927381 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1ec4d-5cb7-43f3-9c73-e0f40a2a52bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:40.927784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1ec4d-5cb7-43f3-9c73-e0f40a2a52bf admin admin] PATH_INFO: `/v3/projects/422340b9e5834d16bff3539c549fc4c1/users/222b727863844d2b95a9c7222a56a94c/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:40.943864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1ec4d-5cb7-43f3-9c73-e0f40a2a52bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=422340b9e5834d16bff3539c549fc4c1, user_id=222b727863844d2b95a9c7222a56a94c, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:40.946606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1ec4d-5cb7-43f3-9c73-e0f40a2a52bf admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:41.006109 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1290/2607] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:43:40 2022] PUT /identity/v3/projects/422340b9e5834d16bff3539c549fc4c1/users/222b727863844d2b95a9c7222a56a94c/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:41.011273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23676-5143-4a9d-892d-484fcd95edbe None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:41.011589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23676-5143-4a9d-892d-484fcd95edbe None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:41.011877 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23676-5143-4a9d-892d-484fcd95edbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:41.029796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-68a23676-5143-4a9d-892d-484fcd95edbe None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:41.031013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-68a23676-5143-4a9d-892d-484fcd95edbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:41.091037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1318/2608] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:41.101666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1ec4d-5cb7-43f3-9c73-e0f40a2a52bf admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:41.103412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c4c1ec4d-5cb7-43f3-9c73-e0f40a2a52bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:41.160955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1ec4d-5cb7-43f3-9c73-e0f40a2a52bf admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:41.164789 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc629f1b-58e8-492c-b756-d03ce5d6fb17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:41.171300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc629f1b-58e8-492c-b756-d03ce5d6fb17 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:41.171300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc629f1b-58e8-492c-b756-d03ce5d6fb17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:41.171300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc629f1b-58e8-492c-b756-d03ce5d6fb17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:41.176452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc629f1b-58e8-492c-b756-d03ce5d6fb17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:41.177977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc629f1b-58e8-492c-b756-d03ce5d6fb17 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:41.188491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1291/2609] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:43:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:43:47.456194 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79007af8-c286-4dc0-a0eb-c5fcf92a32a2 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:47.456715 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79007af8-c286-4dc0-a0eb-c5fcf92a32a2 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:47.456962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79007af8-c286-4dc0-a0eb-c5fcf92a32a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:47.487671 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-79007af8-c286-4dc0-a0eb-c5fcf92a32a2 None None] MFA Rules not processed for user `222b727863844d2b95a9c7222a56a94c`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:47.489150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-79007af8-c286-4dc0-a0eb-c5fcf92a32a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:47.559001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1319/2610] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:47.573275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc629f1b-58e8-492c-b756-d03ce5d6fb17 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:47.574713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-bc629f1b-58e8-492c-b756-d03ce5d6fb17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:47.653375 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc629f1b-58e8-492c-b756-d03ce5d6fb17 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:47.659803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b8c2e-121c-4545-a44f-56fd4cb0a4c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:47.660474 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b8c2e-121c-4545-a44f-56fd4cb0a4c3 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:47.660761 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b8c2e-121c-4545-a44f-56fd4cb0a4c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:47.661140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b8c2e-121c-4545-a44f-56fd4cb0a4c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:47.670906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b8c2e-121c-4545-a44f-56fd4cb0a4c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:47.672557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b8c2e-121c-4545-a44f-56fd4cb0a4c3 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:47.682351 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1292/2611] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:43:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:43:50.102230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c52b9-3e14-4639-ab51-e4483fe0eadd None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:50.102730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c52b9-3e14-4639-ab51-e4483fe0eadd None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:50.102853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c52b9-3e14-4639-ab51-e4483fe0eadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:50.127983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-f51c52b9-3e14-4639-ab51-e4483fe0eadd None None] MFA Rules not processed for user `222b727863844d2b95a9c7222a56a94c`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:50.129216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f51c52b9-3e14-4639-ab51-e4483fe0eadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:50.136583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1320/2612] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:51.374315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b8c2e-121c-4545-a44f-56fd4cb0a4c3 service nova] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:51.378948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a81b8c2e-121c-4545-a44f-56fd4cb0a4c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:51.383913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:51.387924 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b8c2e-121c-4545-a44f-56fd4cb0a4c3 service nova] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:51.392509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee13cb7-2039-4ffb-aadc-121df999ca31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:51.393161 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee13cb7-2039-4ffb-aadc-121df999ca31 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:51.393450 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee13cb7-2039-4ffb-aadc-121df999ca31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:51.393749 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee13cb7-2039-4ffb-aadc-121df999ca31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:51.395586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-bee13cb7-2039-4ffb-aadc-121df999ca31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:51.407944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee13cb7-2039-4ffb-aadc-121df999ca31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:51.409006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee13cb7-2039-4ffb-aadc-121df999ca31 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:51.419134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1293/2613] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:43:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:43:55.429967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c52b9-3e14-4639-ab51-e4483fe0eadd None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:55.431722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f51c52b9-3e14-4639-ab51-e4483fe0eadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:55.446742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c52b9-3e14-4639-ab51-e4483fe0eadd None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:55.451986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:55.452913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:55.453153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:55.453428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] PATH_INFO: `/v3/users/cc8a087e97424d2890952be33695378f` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:55.454645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc8a087e97424d2890952be33695378f)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:55.456630 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:55.668223 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc8a087e97424d2890952be33695378f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:55.746452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc8a087e97424d2890952be33695378f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:55.747243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc8a087e97424d2890952be33695378f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:55.750231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:55.756058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc8a087e97424d2890952be33695378f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:55.756882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8a737d92-6c85-4af4-8963-90bfaf2c1bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc8a087e97424d2890952be33695378f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:55.778361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1321/2614] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:43:55 2022] DELETE /identity/v3/users/cc8a087e97424d2890952be33695378f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:56.069912 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee13cb7-2039-4ffb-aadc-121df999ca31 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:56.080964 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee13cb7-2039-4ffb-aadc-121df999ca31 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:56.144304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:56.145208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.145493 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.145772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] PATH_INFO: `/v3/projects/4b698ead017b4ef8b397ee127b487309` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.147061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b698ead017b4ef8b397ee127b487309)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:56.149015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:56.256019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b698ead017b4ef8b397ee127b487309', 'request_id': 'req-c311bda5-e925-4d61-931b-66da4b5ad703'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:56.262041 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b698ead017b4ef8b397ee127b487309', 'request_id': 'req-c311bda5-e925-4d61-931b-66da4b5ad703'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:56.275143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b698ead017b4ef8b397ee127b487309', 'request_id': 'req-c311bda5-e925-4d61-931b-66da4b5ad703'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:56.351350 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] The token cache is being invalidate because project 4b698ead017b4ef8b397ee127b487309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:43:56.351674 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:56.352293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c311bda5-e925-4d61-931b-66da4b5ad703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:56.354084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1294/2615] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:43:56 2022] DELETE /identity/v3/projects/4b698ead017b4ef8b397ee127b487309 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:56.359268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d89c6-697e-48ae-9c3c-106a64202b3f None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.359589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d89c6-697e-48ae-9c3c-106a64202b3f None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.359871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d89c6-697e-48ae-9c3c-106a64202b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.380535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-b58d89c6-697e-48ae-9c3c-106a64202b3f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:56.381805 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b58d89c6-697e-48ae-9c3c-106a64202b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:56.441700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1322/2616] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:56.449428 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1836cb-a425-42a1-9143-0499b0f65d2d None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.449728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1836cb-a425-42a1-9143-0499b0f65d2d None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.450014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1836cb-a425-42a1-9143-0499b0f65d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.468952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1d1836cb-a425-42a1-9143-0499b0f65d2d None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:56.470156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1d1836cb-a425-42a1-9143-0499b0f65d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:56.478230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1295/2617] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:56.482419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d89c6-697e-48ae-9c3c-106a64202b3f None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:56.492358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d89c6-697e-48ae-9c3c-106a64202b3f None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:56.497073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bb6e2-cc35-4fd5-bf5c-1af55585bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:56.497756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bb6e2-cc35-4fd5-bf5c-1af55585bd4b admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.498044 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bb6e2-cc35-4fd5-bf5c-1af55585bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.498323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bb6e2-cc35-4fd5-bf5c-1af55585bd4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.499557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bb6e2-cc35-4fd5-bf5c-1af55585bd4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:43:56.499875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bb6e2-cc35-4fd5-bf5c-1af55585bd4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:56.501539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bb6e2-cc35-4fd5-bf5c-1af55585bd4b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:56.513222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1323/2618] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:43:56.518093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1836cb-a425-42a1-9143-0499b0f65d2d None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:56.527445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1836cb-a425-42a1-9143-0499b0f65d2d None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:56.532137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ec5b3-c4d7-4fc1-bbc5-25bdc7a794e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:56.532802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ec5b3-c4d7-4fc1-bbc5-25bdc7a794e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.533086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ec5b3-c4d7-4fc1-bbc5-25bdc7a794e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.533368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ec5b3-c4d7-4fc1-bbc5-25bdc7a794e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.534069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ec5b3-c4d7-4fc1-bbc5-25bdc7a794e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:56.536181 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ec5b3-c4d7-4fc1-bbc5-25bdc7a794e9 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:56.617172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1296/2619] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:43:56 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:43:56.621283 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bb6e2-cc35-4fd5-bf5c-1af55585bd4b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:56.632007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bb6e2-cc35-4fd5-bf5c-1af55585bd4b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:56.687467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1adca-4729-407a-a00f-04351c2d2877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:56.688208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1adca-4729-407a-a00f-04351c2d2877 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.688490 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1adca-4729-407a-a00f-04351c2d2877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.688857 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1adca-4729-407a-a00f-04351c2d2877 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.689456 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1adca-4729-407a-a00f-04351c2d2877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:56.691367 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1adca-4729-407a-a00f-04351c2d2877 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:56.770882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1324/2620] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:43:56 2022] POST /identity/v3/users => generated 302 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:43:56.774961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ec5b3-c4d7-4fc1-bbc5-25bdc7a794e9 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:56.785332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ec5b3-c4d7-4fc1-bbc5-25bdc7a794e9 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:56.789901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9a22-2051-4055-88e1-fa2ab5b8b1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:56.790601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9a22-2051-4055-88e1-fa2ab5b8b1ab admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.790882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9a22-2051-4055-88e1-fa2ab5b8b1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.791153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9a22-2051-4055-88e1-fa2ab5b8b1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.791905 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de9a22-2051-4055-88e1-fa2ab5b8b1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:56.793605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de9a22-2051-4055-88e1-fa2ab5b8b1ab admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:56.804720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1297/2621] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:43:56.808649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1adca-4729-407a-a00f-04351c2d2877 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:56.819208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1adca-4729-407a-a00f-04351c2d2877 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:56.823961 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7befda3-14da-4528-b0ce-cce069e9bc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:56.824606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7befda3-14da-4528-b0ce-cce069e9bc84 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.824889 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7befda3-14da-4528-b0ce-cce069e9bc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.825163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7befda3-14da-4528-b0ce-cce069e9bc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.825880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7befda3-14da-4528-b0ce-cce069e9bc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:56.827453 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7befda3-14da-4528-b0ce-cce069e9bc84 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:56.837156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1325/2622] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:43:56.840979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9a22-2051-4055-88e1-fa2ab5b8b1ab admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:56.850562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9a22-2051-4055-88e1-fa2ab5b8b1ab admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:56.855114 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb9c1c-6bce-474a-839c-76d88995c9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:56.856008 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb9c1c-6bce-474a-839c-76d88995c9a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.856293 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb9c1c-6bce-474a-839c-76d88995c9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.856566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb9c1c-6bce-474a-839c-76d88995c9a0 admin admin] PATH_INFO: `/v3/projects/a7db4a74fcf54bcaada2156a296281ee/users/e2ab3182c3434318a32f0871884b23b9/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.872019 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb9c1c-6bce-474a-839c-76d88995c9a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7db4a74fcf54bcaada2156a296281ee, user_id=e2ab3182c3434318a32f0871884b23b9, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:56.875338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb9c1c-6bce-474a-839c-76d88995c9a0 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:56.952464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1298/2623] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:43:56 2022] PUT /identity/v3/projects/a7db4a74fcf54bcaada2156a296281ee/users/e2ab3182c3434318a32f0871884b23b9/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:56.962823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b847f3-402d-4e09-b0f9-da1d7979c9c0 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:56.963117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b847f3-402d-4e09-b0f9-da1d7979c9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:56.963392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b847f3-402d-4e09-b0f9-da1d7979c9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:56.990538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-20b847f3-402d-4e09-b0f9-da1d7979c9c0 None None] MFA Rules not processed for user `e2ab3182c3434318a32f0871884b23b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:56.991815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-20b847f3-402d-4e09-b0f9-da1d7979c9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:57.024202 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:57.059891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1326/2624] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:43:57.065023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb9c1c-6bce-474a-839c-76d88995c9a0 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:57.083973 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:57.126199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb9c1c-6bce-474a-839c-76d88995c9a0 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:57.130955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d94e7a-b288-4d78-a438-436cbbc6ddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:57.131643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d94e7a-b288-4d78-a438-436cbbc6ddfb admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:57.131930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d94e7a-b288-4d78-a438-436cbbc6ddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:57.132204 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d94e7a-b288-4d78-a438-436cbbc6ddfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:57.132932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d94e7a-b288-4d78-a438-436cbbc6ddfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:57.135145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d94e7a-b288-4d78-a438-436cbbc6ddfb admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:57.221505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1299/2625] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:43:57 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:43:57.225533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b847f3-402d-4e09-b0f9-da1d7979c9c0 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:57.237021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b847f3-402d-4e09-b0f9-da1d7979c9c0 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:57.292145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885922d4-220c-4e57-8926-a129802b686f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:57.292815 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885922d4-220c-4e57-8926-a129802b686f admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:57.293094 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885922d4-220c-4e57-8926-a129802b686f admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:57.293366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885922d4-220c-4e57-8926-a129802b686f admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:57.294315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885922d4-220c-4e57-8926-a129802b686f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:57.296938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885922d4-220c-4e57-8926-a129802b686f admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:57.423328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1327/2626] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:43:57 2022] POST /identity/v3/users => generated 309 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:43:57.427299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d94e7a-b288-4d78-a438-436cbbc6ddfb admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:57.437877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d94e7a-b288-4d78-a438-436cbbc6ddfb admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:57.444735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b3346-e626-42e3-8c88-9080b3ead210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:57.449081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b3346-e626-42e3-8c88-9080b3ead210 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:57.449081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b3346-e626-42e3-8c88-9080b3ead210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:57.449081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b3346-e626-42e3-8c88-9080b3ead210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:57.449081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b3346-e626-42e3-8c88-9080b3ead210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:57.449081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b3346-e626-42e3-8c88-9080b3ead210 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:57.458875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1300/2627] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:43:57.463284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885922d4-220c-4e57-8926-a129802b686f admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:57.474052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885922d4-220c-4e57-8926-a129802b686f admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:57.478771 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14f366-e92a-4f02-a6d2-aeaad6592784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:57.479686 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14f366-e92a-4f02-a6d2-aeaad6592784 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:57.479970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14f366-e92a-4f02-a6d2-aeaad6592784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:57.480245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14f366-e92a-4f02-a6d2-aeaad6592784 admin admin] PATH_INFO: `/v3/projects/5bdb230d87ec4fb9adba60639e45f07d/users/b865841e5d804d639fb1aa030844049f/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:57.495307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14f366-e92a-4f02-a6d2-aeaad6592784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bdb230d87ec4fb9adba60639e45f07d, user_id=b865841e5d804d639fb1aa030844049f, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:57.498007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14f366-e92a-4f02-a6d2-aeaad6592784 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:57.622452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1328/2628] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:43:57 2022] PUT /identity/v3/projects/5bdb230d87ec4fb9adba60639e45f07d/users/b865841e5d804d639fb1aa030844049f/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:43:57.626360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b3346-e626-42e3-8c88-9080b3ead210 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:57.636294 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b3346-e626-42e3-8c88-9080b3ead210 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:57.691202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151c9a-6d33-4ee7-99a1-62b3b80fb3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:57.691881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151c9a-6d33-4ee7-99a1-62b3b80fb3bb admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:57.692164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151c9a-6d33-4ee7-99a1-62b3b80fb3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:57.692438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151c9a-6d33-4ee7-99a1-62b3b80fb3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:57.693156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3151c9a-6d33-4ee7-99a1-62b3b80fb3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:57.694739 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3151c9a-6d33-4ee7-99a1-62b3b80fb3bb admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:57.704788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1301/2629] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:43:57.708872 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14f366-e92a-4f02-a6d2-aeaad6592784 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:57.719908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14f366-e92a-4f02-a6d2-aeaad6592784 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:57.724545 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51279e-7f0f-4fdd-bb41-1c5c143f6164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:57.725423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51279e-7f0f-4fdd-bb41-1c5c143f6164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:57.725706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51279e-7f0f-4fdd-bb41-1c5c143f6164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:57.725982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51279e-7f0f-4fdd-bb41-1c5c143f6164 admin admin] PATH_INFO: `/v3/projects/5bdb230d87ec4fb9adba60639e45f07d/users/b865841e5d804d639fb1aa030844049f/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:57.727980 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51279e-7f0f-4fdd-bb41-1c5c143f6164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bdb230d87ec4fb9adba60639e45f07d, user_id=b865841e5d804d639fb1aa030844049f, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:57.730618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51279e-7f0f-4fdd-bb41-1c5c143f6164 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:57.740583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1329/2630] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:43:57 2022] PUT /identity/v3/projects/5bdb230d87ec4fb9adba60639e45f07d/users/b865841e5d804d639fb1aa030844049f/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:57.750342 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decde78e-b452-4b5e-83b3-43e5529c2d81 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:57.750634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decde78e-b452-4b5e-83b3-43e5529c2d81 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:57.750938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decde78e-b452-4b5e-83b3-43e5529c2d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:57.776698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-decde78e-b452-4b5e-83b3-43e5529c2d81 None None] MFA Rules not processed for user `b865841e5d804d639fb1aa030844049f`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:57.777883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-decde78e-b452-4b5e-83b3-43e5529c2d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:57.845463 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1302/2631] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:57 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:57.856108 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51279e-7f0f-4fdd-bb41-1c5c143f6164 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:57.857800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9a51279e-7f0f-4fdd-bb41-1c5c143f6164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:57.915507 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51279e-7f0f-4fdd-bb41-1c5c143f6164 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:57.919446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ab032-f5ab-4a07-9c61-d2b3dc345d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:57.920118 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ab032-f5ab-4a07-9c61-d2b3dc345d09 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:57.920402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ab032-f5ab-4a07-9c61-d2b3dc345d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:57.920679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ab032-f5ab-4a07-9c61-d2b3dc345d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:57.929695 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ab032-f5ab-4a07-9c61-d2b3dc345d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:57.931106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ab032-f5ab-4a07-9c61-d2b3dc345d09 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:57.941049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1330/2632] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:43:57 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:43:58.071939 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decde78e-b452-4b5e-83b3-43e5529c2d81 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:58.085585 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:58.135647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decde78e-b452-4b5e-83b3-43e5529c2d81 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:58.140950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:58.144680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:58.144680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:58.144680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] PATH_INFO: `/v3/users/e2ab3182c3434318a32f0871884b23b9` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:58.144680 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ab3182c3434318a32f0871884b23b9)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:58.145611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:58.327365 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ab3182c3434318a32f0871884b23b9'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:58.328036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ab3182c3434318a32f0871884b23b9'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:58.334860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ab3182c3434318a32f0871884b23b9'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:58.335479 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ab3182c3434318a32f0871884b23b9'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:58.347276 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ab3182c3434318a32f0871884b23b9'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:58.437501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1303/2633] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:43:58 2022] DELETE /identity/v3/users/e2ab3182c3434318a32f0871884b23b9 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:58.442003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7d4fa-a6ae-482e-b435-3ce3a37306eb None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:58.442323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7d4fa-a6ae-482e-b435-3ce3a37306eb None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:58.442607 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7d4fa-a6ae-482e-b435-3ce3a37306eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:58.459989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-8aa7d4fa-a6ae-482e-b435-3ce3a37306eb None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:43:58.461192 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8aa7d4fa-a6ae-482e-b435-3ce3a37306eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:58.484096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:58.524742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1331/2634] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:43:58.534602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:58.536173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:58.592795 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec79a6-d527-4b9c-87b0-9a53d7b3d991 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:58.596560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6b4f-417a-40c3-a928-9f7ade488c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:58.597201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6b4f-417a-40c3-a928-9f7ade488c09 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:58.597511 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6b4f-417a-40c3-a928-9f7ade488c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:58.597810 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6b4f-417a-40c3-a928-9f7ade488c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:58.606752 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f6b4f-417a-40c3-a928-9f7ade488c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:58.608205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f6b4f-417a-40c3-a928-9f7ade488c09 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:58.617937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1304/2635] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:43:59.012155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7d4fa-a6ae-482e-b435-3ce3a37306eb None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:59.013621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-8aa7d4fa-a6ae-482e-b435-3ce3a37306eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:59.027662 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7d4fa-a6ae-482e-b435-3ce3a37306eb None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:59.033250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:59.034989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:59.035356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:59.035665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] PATH_INFO: `/v3/projects/a7db4a74fcf54bcaada2156a296281ee` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:59.036902 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7db4a74fcf54bcaada2156a296281ee)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:59.038915 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:59.129848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7db4a74fcf54bcaada2156a296281ee', 'request_id': 'req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.182074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7db4a74fcf54bcaada2156a296281ee', 'request_id': 'req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.188104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7db4a74fcf54bcaada2156a296281ee', 'request_id': 'req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.199928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] The token cache is being invalidate because project a7db4a74fcf54bcaada2156a296281ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:43:59.200210 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.200928 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.202982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1332/2636] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:43:59 2022] DELETE /identity/v3/projects/a7db4a74fcf54bcaada2156a296281ee => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:43:59.207178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6b4f-417a-40c3-a928-9f7ade488c09 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:59.217118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6b4f-417a-40c3-a928-9f7ade488c09 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:43:59.251086 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:43:59.283990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:43:59.284895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:43:59.285137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:43:59.285412 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] PATH_INFO: `/v3/users/b865841e5d804d639fb1aa030844049f` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:43:59.286686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b865841e5d804d639fb1aa030844049f)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:43:59.288584 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:43:59.443985 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b865841e5d804d639fb1aa030844049f'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.444638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b865841e5d804d639fb1aa030844049f'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.451687 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b865841e5d804d639fb1aa030844049f'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.452867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b865841e5d804d639fb1aa030844049f'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.464581 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-db615723-261f-4958-bf89-95f50c4de8f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b865841e5d804d639fb1aa030844049f'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:43:59.539091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1305/2637] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:43:59 2022] DELETE /identity/v3/users/b865841e5d804d639fb1aa030844049f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:43:59.966369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:43:59.968141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:43:59.982106 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1acee7-4bfb-45d0-8e17-4a667f0f53a1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:00.043159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:00.044308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.044585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.044861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] PATH_INFO: `/v3/projects/5bdb230d87ec4fb9adba60639e45f07d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.046573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bdb230d87ec4fb9adba60639e45f07d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:00.048539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:00.187077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bdb230d87ec4fb9adba60639e45f07d', 'request_id': 'req-b2010dd6-1295-4847-be6b-48b527abd2cc'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:00.188679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:00.253997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bdb230d87ec4fb9adba60639e45f07d', 'request_id': 'req-b2010dd6-1295-4847-be6b-48b527abd2cc'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:00.259839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bdb230d87ec4fb9adba60639e45f07d', 'request_id': 'req-b2010dd6-1295-4847-be6b-48b527abd2cc'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:00.270881 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] The token cache is being invalidate because project 5bdb230d87ec4fb9adba60639e45f07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:00.271217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:00.271960 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b2010dd6-1295-4847-be6b-48b527abd2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:00.273493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1333/2638] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:43:59 2022] DELETE /identity/v3/projects/5bdb230d87ec4fb9adba60639e45f07d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:00.278438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cf53c-20c7-4191-ac8c-04ccb164dde1 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.278814 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cf53c-20c7-4191-ac8c-04ccb164dde1 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.278997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cf53c-20c7-4191-ac8c-04ccb164dde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.298765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8e4cf53c-20c7-4191-ac8c-04ccb164dde1 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:00.299990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8e4cf53c-20c7-4191-ac8c-04ccb164dde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:00.358860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1306/2639] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:00.366326 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe24f1-5e6e-4de7-ace2-57b50b08adc3 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.366611 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe24f1-5e6e-4de7-ace2-57b50b08adc3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.366885 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe24f1-5e6e-4de7-ace2-57b50b08adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.385372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-f6fe24f1-5e6e-4de7-ace2-57b50b08adc3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:00.386561 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f6fe24f1-5e6e-4de7-ace2-57b50b08adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:00.394713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1334/2640] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:00.398777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cf53c-20c7-4191-ac8c-04ccb164dde1 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:00.409800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cf53c-20c7-4191-ac8c-04ccb164dde1 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:00.414889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00be768-a3b8-48f1-b60d-586fc42cd091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:00.415661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00be768-a3b8-48f1-b60d-586fc42cd091 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.415950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00be768-a3b8-48f1-b60d-586fc42cd091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.417394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00be768-a3b8-48f1-b60d-586fc42cd091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.417394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00be768-a3b8-48f1-b60d-586fc42cd091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:44:00.417394 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00be768-a3b8-48f1-b60d-586fc42cd091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:00.419244 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00be768-a3b8-48f1-b60d-586fc42cd091 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:00.433360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1307/2641] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:44:00.441757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe24f1-5e6e-4de7-ace2-57b50b08adc3 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:00.447260 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe24f1-5e6e-4de7-ace2-57b50b08adc3 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:00.452066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1d7e9-5105-4b7d-b5f6-24bd675ea944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:00.452724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1d7e9-5105-4b7d-b5f6-24bd675ea944 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.453023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1d7e9-5105-4b7d-b5f6-24bd675ea944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.453434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1d7e9-5105-4b7d-b5f6-24bd675ea944 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.454175 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1d7e9-5105-4b7d-b5f6-24bd675ea944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:00.456112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1d7e9-5105-4b7d-b5f6-24bd675ea944 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:00.544023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1335/2642] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:00 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:44:00.548063 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00be768-a3b8-48f1-b60d-586fc42cd091 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:00.558167 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00be768-a3b8-48f1-b60d-586fc42cd091 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:00.605684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:00.621433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fd3dd-7d10-4a87-b2ce-e97c96c8e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:00.622103 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fd3dd-7d10-4a87-b2ce-e97c96c8e89e admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.622389 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fd3dd-7d10-4a87-b2ce-e97c96c8e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.622665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fd3dd-7d10-4a87-b2ce-e97c96c8e89e admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.623367 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fd3dd-7d10-4a87-b2ce-e97c96c8e89e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:00.625393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fd3dd-7d10-4a87-b2ce-e97c96c8e89e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:00.719579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1308/2643] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:00 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:00.723602 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1d7e9-5105-4b7d-b5f6-24bd675ea944 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:00.733762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1d7e9-5105-4b7d-b5f6-24bd675ea944 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:00.738329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5020-a52b-4010-b5f9-197e5cfb30c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:00.738973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5020-a52b-4010-b5f9-197e5cfb30c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.739253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5020-a52b-4010-b5f9-197e5cfb30c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.739522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5020-a52b-4010-b5f9-197e5cfb30c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.740269 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b5020-a52b-4010-b5f9-197e5cfb30c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:00.741845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b5020-a52b-4010-b5f9-197e5cfb30c9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:00.752009 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1336/2644] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:00.755724 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fd3dd-7d10-4a87-b2ce-e97c96c8e89e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:00.766072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fd3dd-7d10-4a87-b2ce-e97c96c8e89e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:00.770757 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772fd47-d0f2-449e-842d-533deb818d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:00.771409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772fd47-d0f2-449e-842d-533deb818d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.771732 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772fd47-d0f2-449e-842d-533deb818d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.772010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772fd47-d0f2-449e-842d-533deb818d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.772733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772fd47-d0f2-449e-842d-533deb818d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:00.774331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772fd47-d0f2-449e-842d-533deb818d70 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:00.784874 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1309/2645] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:00.788853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5020-a52b-4010-b5f9-197e5cfb30c9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:00.799027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5020-a52b-4010-b5f9-197e5cfb30c9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:00.803705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a4235-9304-4c59-acda-eb527004c196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:00.804571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a4235-9304-4c59-acda-eb527004c196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.804851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a4235-9304-4c59-acda-eb527004c196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.805124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a4235-9304-4c59-acda-eb527004c196 admin admin] PATH_INFO: `/v3/projects/37f654a6b1674b4a9d1ce32cf83d4160/users/659b79a5fc88422cb88dcd49325c7ffb/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.819238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a4235-9304-4c59-acda-eb527004c196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f654a6b1674b4a9d1ce32cf83d4160, user_id=659b79a5fc88422cb88dcd49325c7ffb, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:00.821926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a4235-9304-4c59-acda-eb527004c196 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:00.888858 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1337/2646] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:00 2022] PUT /identity/v3/projects/37f654a6b1674b4a9d1ce32cf83d4160/users/659b79a5fc88422cb88dcd49325c7ffb/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:44:00.900873 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c6679-e597-4a78-9d2d-a52089306172 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:00.901201 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c6679-e597-4a78-9d2d-a52089306172 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:00.901485 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c6679-e597-4a78-9d2d-a52089306172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:00.928028 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-018c6679-e597-4a78-9d2d-a52089306172 None None] MFA Rules not processed for user `659b79a5fc88422cb88dcd49325c7ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:00.929228 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-018c6679-e597-4a78-9d2d-a52089306172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:00.992062 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1310/2647] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:00.996474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a4235-9304-4c59-acda-eb527004c196 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:01.062355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a4235-9304-4c59-acda-eb527004c196 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:01.068700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868aea80-feef-45bb-a5e7-ff58c82f370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:01.069446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868aea80-feef-45bb-a5e7-ff58c82f370c admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:01.069727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868aea80-feef-45bb-a5e7-ff58c82f370c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:01.070003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868aea80-feef-45bb-a5e7-ff58c82f370c admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:01.070685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868aea80-feef-45bb-a5e7-ff58c82f370c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:01.072626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868aea80-feef-45bb-a5e7-ff58c82f370c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:01.149076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1338/2648] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:00 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:01.153216 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c6679-e597-4a78-9d2d-a52089306172 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:01.163798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c6679-e597-4a78-9d2d-a52089306172 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:01.218901 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b16b1-d2d8-482e-8918-b4e190146eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:01.219583 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b16b1-d2d8-482e-8918-b4e190146eca admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:01.219866 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b16b1-d2d8-482e-8918-b4e190146eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:01.220141 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b16b1-d2d8-482e-8918-b4e190146eca admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:01.220818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b16b1-d2d8-482e-8918-b4e190146eca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:01.222717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b16b1-d2d8-482e-8918-b4e190146eca admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:01.308277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1311/2649] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:01 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:01.313264 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868aea80-feef-45bb-a5e7-ff58c82f370c admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:01.323971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868aea80-feef-45bb-a5e7-ff58c82f370c admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:01.328720 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa7202-b5a0-47a6-90a3-336c052f0a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:01.329424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa7202-b5a0-47a6-90a3-336c052f0a05 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:01.329706 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa7202-b5a0-47a6-90a3-336c052f0a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:01.329983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa7202-b5a0-47a6-90a3-336c052f0a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:01.330719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa7202-b5a0-47a6-90a3-336c052f0a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:01.332406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa7202-b5a0-47a6-90a3-336c052f0a05 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:01.342848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1339/2650] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:01.346865 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b16b1-d2d8-482e-8918-b4e190146eca admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:01.356968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b16b1-d2d8-482e-8918-b4e190146eca admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:01.362139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b6317-2afd-4845-8f2d-5069285255d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:01.363015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b6317-2afd-4845-8f2d-5069285255d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:01.363309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b6317-2afd-4845-8f2d-5069285255d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:01.363601 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b6317-2afd-4845-8f2d-5069285255d7 admin admin] PATH_INFO: `/v3/projects/3095181ed2054e53b55d09c8e97860bd/users/d949426fe393474e9503f76ac6d19cb1/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:01.378845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b6317-2afd-4845-8f2d-5069285255d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3095181ed2054e53b55d09c8e97860bd, user_id=d949426fe393474e9503f76ac6d19cb1, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:01.381560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b6317-2afd-4845-8f2d-5069285255d7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:01.468518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1312/2651] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:01 2022] PUT /identity/v3/projects/3095181ed2054e53b55d09c8e97860bd/users/d949426fe393474e9503f76ac6d19cb1/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:01.472619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa7202-b5a0-47a6-90a3-336c052f0a05 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:01.482765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa7202-b5a0-47a6-90a3-336c052f0a05 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:01.528642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:01.541209 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1850f6-8c03-4a94-8c28-666b39091f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:01.541879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1850f6-8c03-4a94-8c28-666b39091f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:01.542160 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1850f6-8c03-4a94-8c28-666b39091f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:01.542475 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1850f6-8c03-4a94-8c28-666b39091f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:01.543226 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1850f6-8c03-4a94-8c28-666b39091f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:01.545113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1850f6-8c03-4a94-8c28-666b39091f44 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:01.555987 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1340/2652] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:01.560305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b6317-2afd-4845-8f2d-5069285255d7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:01.571500 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b6317-2afd-4845-8f2d-5069285255d7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:01.576387 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48ff8d-0d5d-40e7-9f03-6a4beee15a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:01.577294 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48ff8d-0d5d-40e7-9f03-6a4beee15a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:01.577577 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48ff8d-0d5d-40e7-9f03-6a4beee15a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:01.577897 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48ff8d-0d5d-40e7-9f03-6a4beee15a71 admin admin] PATH_INFO: `/v3/projects/3095181ed2054e53b55d09c8e97860bd/users/d949426fe393474e9503f76ac6d19cb1/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:01.580015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48ff8d-0d5d-40e7-9f03-6a4beee15a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3095181ed2054e53b55d09c8e97860bd, user_id=d949426fe393474e9503f76ac6d19cb1, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:01.582828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48ff8d-0d5d-40e7-9f03-6a4beee15a71 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:01.596802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1313/2653] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:01 2022] PUT /identity/v3/projects/3095181ed2054e53b55d09c8e97860bd/users/d949426fe393474e9503f76ac6d19cb1/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:44:01.606897 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea77a1-66d3-419e-80fb-5596d6c8fb2e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:01.610501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea77a1-66d3-419e-80fb-5596d6c8fb2e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:01.610501 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea77a1-66d3-419e-80fb-5596d6c8fb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:01.637203 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-30ea77a1-66d3-419e-80fb-5596d6c8fb2e None None] MFA Rules not processed for user `d949426fe393474e9503f76ac6d19cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:01.639008 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-30ea77a1-66d3-419e-80fb-5596d6c8fb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:01.721153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1341/2654] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:01 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:01.732146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48ff8d-0d5d-40e7-9f03-6a4beee15a71 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:01.734452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-de48ff8d-0d5d-40e7-9f03-6a4beee15a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:01.740067 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:01.797003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48ff8d-0d5d-40e7-9f03-6a4beee15a71 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:01.800903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5a91d-044d-45ae-8661-1277f91e9fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:01.801556 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5a91d-044d-45ae-8661-1277f91e9fba service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:01.801838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5a91d-044d-45ae-8661-1277f91e9fba service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:01.802120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5a91d-044d-45ae-8661-1277f91e9fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:01.811460 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5a91d-044d-45ae-8661-1277f91e9fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:01.812918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5a91d-044d-45ae-8661-1277f91e9fba service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:01.822518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1314/2655] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:02.075727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea77a1-66d3-419e-80fb-5596d6c8fb2e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:02.085903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea77a1-66d3-419e-80fb-5596d6c8fb2e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:02.089463 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d98df-a8f8-4dbe-82ec-79f86e604b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:02.090099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d98df-a8f8-4dbe-82ec-79f86e604b0f service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:02.090377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d98df-a8f8-4dbe-82ec-79f86e604b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:02.090647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d98df-a8f8-4dbe-82ec-79f86e604b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:02.146804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d98df-a8f8-4dbe-82ec-79f86e604b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:02.148282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d98df-a8f8-4dbe-82ec-79f86e604b0f service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:02.157835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1342/2656] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:05.597129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5a91d-044d-45ae-8661-1277f91e9fba service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:05.671845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5a91d-044d-45ae-8661-1277f91e9fba service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:05.677415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:05.678579 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:05.679092 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:05.679608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] PATH_INFO: `/v3/users/659b79a5fc88422cb88dcd49325c7ffb` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:05.681781 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=659b79a5fc88422cb88dcd49325c7ffb)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:05.684149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:05.928907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659b79a5fc88422cb88dcd49325c7ffb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:05.929437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659b79a5fc88422cb88dcd49325c7ffb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:05.937415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659b79a5fc88422cb88dcd49325c7ffb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:05.938055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659b79a5fc88422cb88dcd49325c7ffb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:05.950833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659b79a5fc88422cb88dcd49325c7ffb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:06.030269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1315/2657] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:44:05 2022] DELETE /identity/v3/users/659b79a5fc88422cb88dcd49325c7ffb => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:06.030412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119913f4-67c4-4399-a865-352be1f42e04 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:06.030412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119913f4-67c4-4399-a865-352be1f42e04 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:06.030412 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119913f4-67c4-4399-a865-352be1f42e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:06.047279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-119913f4-67c4-4399-a865-352be1f42e04 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:06.048664 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-119913f4-67c4-4399-a865-352be1f42e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:06.113687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1343/2658] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:06.124467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:06.126311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:06.157669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:06.188309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17f4a0-796c-46df-b19d-f3b733dfc337 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:06.191998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86743ec1-525d-41cf-9e77-b153c68ddd01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:06.192648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86743ec1-525d-41cf-9e77-b153c68ddd01 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:06.192935 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86743ec1-525d-41cf-9e77-b153c68ddd01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:06.193213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86743ec1-525d-41cf-9e77-b153c68ddd01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:06.204593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86743ec1-525d-41cf-9e77-b153c68ddd01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:06.206033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86743ec1-525d-41cf-9e77-b153c68ddd01 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:06.215807 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1316/2659] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:06.638758 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119913f4-67c4-4399-a865-352be1f42e04 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:06.640812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-119913f4-67c4-4399-a865-352be1f42e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:06.652950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119913f4-67c4-4399-a865-352be1f42e04 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:06.658514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:06.659393 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:06.659708 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:06.659989 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] PATH_INFO: `/v3/projects/37f654a6b1674b4a9d1ce32cf83d4160` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:06.661197 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f654a6b1674b4a9d1ce32cf83d4160)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:06.663123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:06.667910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:06.752658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f654a6b1674b4a9d1ce32cf83d4160', 'request_id': 'req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:06.844629 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f654a6b1674b4a9d1ce32cf83d4160', 'request_id': 'req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:06.850495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f654a6b1674b4a9d1ce32cf83d4160', 'request_id': 'req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:06.861521 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] The token cache is being invalidate because project 37f654a6b1674b4a9d1ce32cf83d4160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:06.861809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:06.862439 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:06.864029 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1344/2660] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:44:06 2022] DELETE /identity/v3/projects/37f654a6b1674b4a9d1ce32cf83d4160 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:06.868200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86743ec1-525d-41cf-9e77-b153c68ddd01 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:06.878153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86743ec1-525d-41cf-9e77-b153c68ddd01 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:06.932420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:06.933269 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:06.933546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:06.933819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] PATH_INFO: `/v3/users/d949426fe393474e9503f76ac6d19cb1` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:06.934989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d949426fe393474e9503f76ac6d19cb1)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:06.937434 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:07.190076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd949426fe393474e9503f76ac6d19cb1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.190759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd949426fe393474e9503f76ac6d19cb1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.197845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd949426fe393474e9503f76ac6d19cb1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.198483 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd949426fe393474e9503f76ac6d19cb1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.210820 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c27a10bf-9b6d-4e34-bfa4-0a561b5a1e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd949426fe393474e9503f76ac6d19cb1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.267774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1317/2661] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:44:06 2022] DELETE /identity/v3/users/d949426fe393474e9503f76ac6d19cb1 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:44:07.680719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:07.682198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:07.695442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ff35e-5a9d-429f-b5ea-f2fce4ad7134 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:07.753145 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:07.754012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:07.754298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:07.754577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] PATH_INFO: `/v3/projects/3095181ed2054e53b55d09c8e97860bd` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:07.755851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3095181ed2054e53b55d09c8e97860bd)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:07.757834 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:07.856641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3095181ed2054e53b55d09c8e97860bd', 'request_id': 'req-c4b5bc20-be6d-415a-a554-36e420c37362'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.939916 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3095181ed2054e53b55d09c8e97860bd', 'request_id': 'req-c4b5bc20-be6d-415a-a554-36e420c37362'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.945614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3095181ed2054e53b55d09c8e97860bd', 'request_id': 'req-c4b5bc20-be6d-415a-a554-36e420c37362'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.957233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] The token cache is being invalidate because project 3095181ed2054e53b55d09c8e97860bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:07.957461 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.958097 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4b5bc20-be6d-415a-a554-36e420c37362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:07.959689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1345/2662] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:44:07 2022] DELETE /identity/v3/projects/3095181ed2054e53b55d09c8e97860bd => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:07.964558 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3538818-f858-4437-8a5f-aaeeae234ff8 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:07.965030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3538818-f858-4437-8a5f-aaeeae234ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:07.966416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3538818-f858-4437-8a5f-aaeeae234ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:07.984341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a3538818-f858-4437-8a5f-aaeeae234ff8 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:07.985545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a3538818-f858-4437-8a5f-aaeeae234ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:08.026711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:08.044693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1318/2663] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:08.052583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3fa70-1603-424d-a980-44c3bb746b53 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.052878 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3fa70-1603-424d-a980-44c3bb746b53 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.053183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3fa70-1603-424d-a980-44c3bb746b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.070534 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-61b3fa70-1603-424d-a980-44c3bb746b53 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:08.071783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-61b3fa70-1603-424d-a980-44c3bb746b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:08.080237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1346/2664] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:08.084253 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3538818-f858-4437-8a5f-aaeeae234ff8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:08.094267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3538818-f858-4437-8a5f-aaeeae234ff8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:08.098938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788eb1f-0706-4fe1-9e63-646a980b1783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:08.099641 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788eb1f-0706-4fe1-9e63-646a980b1783 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.099931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788eb1f-0706-4fe1-9e63-646a980b1783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.100211 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788eb1f-0706-4fe1-9e63-646a980b1783 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.100937 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788eb1f-0706-4fe1-9e63-646a980b1783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:44:08.101247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788eb1f-0706-4fe1-9e63-646a980b1783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:08.102856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788eb1f-0706-4fe1-9e63-646a980b1783 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:08.114816 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1319/2665] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:44:08.118963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3fa70-1603-424d-a980-44c3bb746b53 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:08.128487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3fa70-1603-424d-a980-44c3bb746b53 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:08.133126 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf8d28-b555-4c85-bb50-b47ce4f933f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:08.133772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf8d28-b555-4c85-bb50-b47ce4f933f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.134056 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf8d28-b555-4c85-bb50-b47ce4f933f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.134333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf8d28-b555-4c85-bb50-b47ce4f933f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.135023 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf8d28-b555-4c85-bb50-b47ce4f933f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:08.136954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf8d28-b555-4c85-bb50-b47ce4f933f9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:08.150576 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:08.222050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1347/2666] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:08 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:44:08.226258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788eb1f-0706-4fe1-9e63-646a980b1783 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:08.236532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788eb1f-0706-4fe1-9e63-646a980b1783 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:08.293091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28427aab-4fbf-42e3-9d4e-995e62e8abf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:08.293761 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28427aab-4fbf-42e3-9d4e-995e62e8abf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.294295 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28427aab-4fbf-42e3-9d4e-995e62e8abf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.294570 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28427aab-4fbf-42e3-9d4e-995e62e8abf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.295271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28427aab-4fbf-42e3-9d4e-995e62e8abf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:08.297267 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28427aab-4fbf-42e3-9d4e-995e62e8abf7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:08.368285 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1320/2667] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:08 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:08.372515 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf8d28-b555-4c85-bb50-b47ce4f933f9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:08.383435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf8d28-b555-4c85-bb50-b47ce4f933f9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:08.388077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3c91d-5d1e-4fc5-a210-d511194dbe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:08.388986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3c91d-5d1e-4fc5-a210-d511194dbe03 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.388986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3c91d-5d1e-4fc5-a210-d511194dbe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.389860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3c91d-5d1e-4fc5-a210-d511194dbe03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.390714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3c91d-5d1e-4fc5-a210-d511194dbe03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:08.392344 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3c91d-5d1e-4fc5-a210-d511194dbe03 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:08.402549 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1348/2668] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:08.407786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28427aab-4fbf-42e3-9d4e-995e62e8abf7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:08.417547 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28427aab-4fbf-42e3-9d4e-995e62e8abf7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:08.422384 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c05554-5440-4edd-b7f6-a52081f73a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:08.423027 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c05554-5440-4edd-b7f6-a52081f73a4f admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.423309 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c05554-5440-4edd-b7f6-a52081f73a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.423614 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c05554-5440-4edd-b7f6-a52081f73a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.424334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c05554-5440-4edd-b7f6-a52081f73a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:08.425931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c05554-5440-4edd-b7f6-a52081f73a4f admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:08.436692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1321/2669] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:08.440738 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3c91d-5d1e-4fc5-a210-d511194dbe03 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:08.450589 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3c91d-5d1e-4fc5-a210-d511194dbe03 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:08.455898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d426a1-f829-4dab-a597-b21489aefb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:08.456764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d426a1-f829-4dab-a597-b21489aefb63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.457049 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d426a1-f829-4dab-a597-b21489aefb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.457327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d426a1-f829-4dab-a597-b21489aefb63 admin admin] PATH_INFO: `/v3/projects/b6cc9f0381564a4abbdbd7aaf461ca33/users/fd493184313049d9a8ddb70d75214f29/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.471779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d426a1-f829-4dab-a597-b21489aefb63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6cc9f0381564a4abbdbd7aaf461ca33, user_id=fd493184313049d9a8ddb70d75214f29, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:08.474378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d426a1-f829-4dab-a597-b21489aefb63 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:08.540195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1349/2670] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:08 2022] PUT /identity/v3/projects/b6cc9f0381564a4abbdbd7aaf461ca33/users/fd493184313049d9a8ddb70d75214f29/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:08.550245 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62cfcc-33a7-40b0-bab8-d1401935be54 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.550568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62cfcc-33a7-40b0-bab8-d1401935be54 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.550872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62cfcc-33a7-40b0-bab8-d1401935be54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.577944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-ac62cfcc-33a7-40b0-bab8-d1401935be54 None None] MFA Rules not processed for user `fd493184313049d9a8ddb70d75214f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:08.579198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-ac62cfcc-33a7-40b0-bab8-d1401935be54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:08.641831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1322/2671] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:08.646217 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d426a1-f829-4dab-a597-b21489aefb63 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:08.707450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d426a1-f829-4dab-a597-b21489aefb63 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:08.712114 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deba2f4-17a9-4554-b6ab-4c5abb86a14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:08.712761 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deba2f4-17a9-4554-b6ab-4c5abb86a14e admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.713041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deba2f4-17a9-4554-b6ab-4c5abb86a14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.713313 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deba2f4-17a9-4554-b6ab-4c5abb86a14e admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.713996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deba2f4-17a9-4554-b6ab-4c5abb86a14e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:08.715931 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deba2f4-17a9-4554-b6ab-4c5abb86a14e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:08.804518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1350/2672] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:08 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:44:08.808700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62cfcc-33a7-40b0-bab8-d1401935be54 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:08.820397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62cfcc-33a7-40b0-bab8-d1401935be54 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:08.868364 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:08.889708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b9841-87f4-41c1-bc78-129df9d7b944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:08.890392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b9841-87f4-41c1-bc78-129df9d7b944 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:08.890675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b9841-87f4-41c1-bc78-129df9d7b944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:08.890953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b9841-87f4-41c1-bc78-129df9d7b944 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:08.891694 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b9841-87f4-41c1-bc78-129df9d7b944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:08.893672 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b9841-87f4-41c1-bc78-129df9d7b944 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:08.982335 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1323/2673] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:08 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:08.987905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deba2f4-17a9-4554-b6ab-4c5abb86a14e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:08.998354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deba2f4-17a9-4554-b6ab-4c5abb86a14e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:09.003757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549bf88-20c7-4fe8-beaf-5de454900720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:09.004417 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549bf88-20c7-4fe8-beaf-5de454900720 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.004701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549bf88-20c7-4fe8-beaf-5de454900720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.004979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549bf88-20c7-4fe8-beaf-5de454900720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.005730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549bf88-20c7-4fe8-beaf-5de454900720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:09.007331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549bf88-20c7-4fe8-beaf-5de454900720 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:09.017882 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1351/2674] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:09.021903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b9841-87f4-41c1-bc78-129df9d7b944 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:09.031849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b9841-87f4-41c1-bc78-129df9d7b944 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:09.036741 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55af681-de9c-4c4d-a9c7-766d31c946b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:09.037393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55af681-de9c-4c4d-a9c7-766d31c946b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.037678 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55af681-de9c-4c4d-a9c7-766d31c946b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.037955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55af681-de9c-4c4d-a9c7-766d31c946b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.038681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55af681-de9c-4c4d-a9c7-766d31c946b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:09.040401 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55af681-de9c-4c4d-a9c7-766d31c946b3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:09.051407 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1324/2675] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:44:09.055449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549bf88-20c7-4fe8-beaf-5de454900720 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:09.065951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549bf88-20c7-4fe8-beaf-5de454900720 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:09.070559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735f531-9981-4513-b96b-d7d3c18168de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:09.071418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735f531-9981-4513-b96b-d7d3c18168de admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.071786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735f531-9981-4513-b96b-d7d3c18168de admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.072058 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735f531-9981-4513-b96b-d7d3c18168de admin admin] PATH_INFO: `/v3/projects/f64bd3d377694138b755fd58a6c36564/users/e4639fe08506403ab4bdeb7d61fc05eb/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.087229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735f531-9981-4513-b96b-d7d3c18168de admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64bd3d377694138b755fd58a6c36564, user_id=e4639fe08506403ab4bdeb7d61fc05eb, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:09.089972 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735f531-9981-4513-b96b-d7d3c18168de admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:09.163518 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1352/2676] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:09 2022] PUT /identity/v3/projects/f64bd3d377694138b755fd58a6c36564/users/e4639fe08506403ab4bdeb7d61fc05eb/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:09.174574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7277ae-92ed-4a69-9ecd-4e052b66d9e8 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.174911 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7277ae-92ed-4a69-9ecd-4e052b66d9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.175198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7277ae-92ed-4a69-9ecd-4e052b66d9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.204696 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9b7277ae-92ed-4a69-9ecd-4e052b66d9e8 None None] MFA Rules not processed for user `e4639fe08506403ab4bdeb7d61fc05eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:09.205934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9b7277ae-92ed-4a69-9ecd-4e052b66d9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:09.270018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1325/2677] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:09.274911 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735f531-9981-4513-b96b-d7d3c18168de admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:09.341206 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735f531-9981-4513-b96b-d7d3c18168de admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:09.346404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145ef23-253c-4766-a2e3-bd81a686f8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:09.347060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145ef23-253c-4766-a2e3-bd81a686f8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.347343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145ef23-253c-4766-a2e3-bd81a686f8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.347655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145ef23-253c-4766-a2e3-bd81a686f8e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.348356 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145ef23-253c-4766-a2e3-bd81a686f8e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:09.350272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145ef23-253c-4766-a2e3-bd81a686f8e5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:09.363704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:09.436728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1353/2678] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:09 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:44:09.440488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7277ae-92ed-4a69-9ecd-4e052b66d9e8 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:09.451399 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7277ae-92ed-4a69-9ecd-4e052b66d9e8 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:09.506956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a34991-614a-47ed-a65d-51a7ceeba015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:09.507667 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a34991-614a-47ed-a65d-51a7ceeba015 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.507956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a34991-614a-47ed-a65d-51a7ceeba015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.508230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a34991-614a-47ed-a65d-51a7ceeba015 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.508929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a34991-614a-47ed-a65d-51a7ceeba015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:09.510847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a34991-614a-47ed-a65d-51a7ceeba015 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:09.600458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1326/2679] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:09 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:09.604245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145ef23-253c-4766-a2e3-bd81a686f8e5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:09.614621 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145ef23-253c-4766-a2e3-bd81a686f8e5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:09.620099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9740d0b-42d2-4a56-80ba-0b8a31d215d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:09.620892 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9740d0b-42d2-4a56-80ba-0b8a31d215d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.621331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9740d0b-42d2-4a56-80ba-0b8a31d215d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.621814 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9740d0b-42d2-4a56-80ba-0b8a31d215d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.622714 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9740d0b-42d2-4a56-80ba-0b8a31d215d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:09.624517 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9740d0b-42d2-4a56-80ba-0b8a31d215d9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:09.635568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1354/2680] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:09.639959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a34991-614a-47ed-a65d-51a7ceeba015 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:09.649982 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a34991-614a-47ed-a65d-51a7ceeba015 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:09.654734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18510c5-b867-4419-ba29-61ae86212c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:09.656232 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18510c5-b867-4419-ba29-61ae86212c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.656524 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18510c5-b867-4419-ba29-61ae86212c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.656802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18510c5-b867-4419-ba29-61ae86212c89 admin admin] PATH_INFO: `/v3/projects/28d3d6db37794c7d948147c37dd4eb81/users/ec2f775946ec49888c5fe8e24738b22d/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.671898 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18510c5-b867-4419-ba29-61ae86212c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28d3d6db37794c7d948147c37dd4eb81, user_id=ec2f775946ec49888c5fe8e24738b22d, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:09.674568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18510c5-b867-4419-ba29-61ae86212c89 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:09.731574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1327/2681] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:09 2022] PUT /identity/v3/projects/28d3d6db37794c7d948147c37dd4eb81/users/ec2f775946ec49888c5fe8e24738b22d/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:09.735994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9740d0b-42d2-4a56-80ba-0b8a31d215d9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:09.746302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9740d0b-42d2-4a56-80ba-0b8a31d215d9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:09.802177 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4284970-f3ba-43d5-922b-2a47080872ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:09.802841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4284970-f3ba-43d5-922b-2a47080872ec admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.803123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4284970-f3ba-43d5-922b-2a47080872ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.803396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4284970-f3ba-43d5-922b-2a47080872ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.804155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4284970-f3ba-43d5-922b-2a47080872ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:09.805749 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4284970-f3ba-43d5-922b-2a47080872ec admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:09.816705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1355/2682] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:09.820956 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18510c5-b867-4419-ba29-61ae86212c89 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:09.832172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18510c5-b867-4419-ba29-61ae86212c89 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:09.836969 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b85064-cefc-46a4-a3e3-6ec6fef97581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:09.837857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b85064-cefc-46a4-a3e3-6ec6fef97581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.838145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b85064-cefc-46a4-a3e3-6ec6fef97581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.838422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b85064-cefc-46a4-a3e3-6ec6fef97581 admin admin] PATH_INFO: `/v3/projects/28d3d6db37794c7d948147c37dd4eb81/users/ec2f775946ec49888c5fe8e24738b22d/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.840586 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b85064-cefc-46a4-a3e3-6ec6fef97581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28d3d6db37794c7d948147c37dd4eb81, user_id=ec2f775946ec49888c5fe8e24738b22d, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:09.843886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b85064-cefc-46a4-a3e3-6ec6fef97581 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:09.854438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1328/2683] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:09 2022] PUT /identity/v3/projects/28d3d6db37794c7d948147c37dd4eb81/users/ec2f775946ec49888c5fe8e24738b22d/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:44:09.865775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447381d2-9335-4408-b9ee-cf526a1f8127 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:09.866071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447381d2-9335-4408-b9ee-cf526a1f8127 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:09.866483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447381d2-9335-4408-b9ee-cf526a1f8127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:09.897977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-447381d2-9335-4408-b9ee-cf526a1f8127 None None] MFA Rules not processed for user `ec2f775946ec49888c5fe8e24738b22d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:09.899241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-447381d2-9335-4408-b9ee-cf526a1f8127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:09.968918 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1356/2684] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:09.980458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b85064-cefc-46a4-a3e3-6ec6fef97581 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:09.982172 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-35b85064-cefc-46a4-a3e3-6ec6fef97581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:09.987758 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:10.040202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b85064-cefc-46a4-a3e3-6ec6fef97581 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:10.044261 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e1280-ba89-43b4-bd26-ecabb84b1eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:10.044849 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e1280-ba89-43b4-bd26-ecabb84b1eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:10.045200 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e1280-ba89-43b4-bd26-ecabb84b1eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:10.045481 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e1280-ba89-43b4-bd26-ecabb84b1eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:10.055065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e1280-ba89-43b4-bd26-ecabb84b1eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:10.056573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e1280-ba89-43b4-bd26-ecabb84b1eb7 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:10.066729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1329/2685] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:10.369498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447381d2-9335-4408-b9ee-cf526a1f8127 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:10.370901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-447381d2-9335-4408-b9ee-cf526a1f8127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:10.383319 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447381d2-9335-4408-b9ee-cf526a1f8127 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:10.386906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3182507-407b-49e7-815d-1cce914dc3ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:10.387560 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3182507-407b-49e7-815d-1cce914dc3ee service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:10.387847 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3182507-407b-49e7-815d-1cce914dc3ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:10.388117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3182507-407b-49e7-815d-1cce914dc3ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:10.445050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3182507-407b-49e7-815d-1cce914dc3ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:10.446508 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3182507-407b-49e7-815d-1cce914dc3ee service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:10.456137 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1357/2686] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:14.514618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e1280-ba89-43b4-bd26-ecabb84b1eb7 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:14.576747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e1280-ba89-43b4-bd26-ecabb84b1eb7 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:14.581925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:14.582784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:14.583072 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:14.583352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] PATH_INFO: `/v3/users/fd493184313049d9a8ddb70d75214f29` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:14.584575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd493184313049d9a8ddb70d75214f29)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:14.586551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:14.781868 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd493184313049d9a8ddb70d75214f29'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:14.782542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd493184313049d9a8ddb70d75214f29'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:14.789398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd493184313049d9a8ddb70d75214f29'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:14.790052 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd493184313049d9a8ddb70d75214f29'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:14.802813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd493184313049d9a8ddb70d75214f29'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:14.873446 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1330/2687] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:44:14 2022] DELETE /identity/v3/users/fd493184313049d9a8ddb70d75214f29 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:14.878537 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f40dfa-7400-4700-84ec-cc92d2492f06 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:14.879011 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f40dfa-7400-4700-84ec-cc92d2492f06 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:14.879138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f40dfa-7400-4700-84ec-cc92d2492f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:14.884765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:14.897195 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-f5f40dfa-7400-4700-84ec-cc92d2492f06 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:14.898402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f5f40dfa-7400-4700-84ec-cc92d2492f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:14.956375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1358/2688] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:14.966149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:14.967747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:14.995802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:15.024273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f00bb-d41c-4004-b2b3-3c95ae879805 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:15.028045 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c6910-1d93-4743-a4b1-f000535913f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:15.035635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c6910-1d93-4743-a4b1-f000535913f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:15.035635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c6910-1d93-4743-a4b1-f000535913f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:15.035635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c6910-1d93-4743-a4b1-f000535913f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:15.038227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c6910-1d93-4743-a4b1-f000535913f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:15.039772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c6910-1d93-4743-a4b1-f000535913f8 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:15.050044 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1331/2689] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:15.451999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f40dfa-7400-4700-84ec-cc92d2492f06 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:15.455933 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-f5f40dfa-7400-4700-84ec-cc92d2492f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:15.465850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f40dfa-7400-4700-84ec-cc92d2492f06 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:15.470974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:15.471871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:15.472155 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:15.472891 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] PATH_INFO: `/v3/projects/b6cc9f0381564a4abbdbd7aaf461ca33` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:15.473683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6cc9f0381564a4abbdbd7aaf461ca33)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:15.475631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:15.565348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6cc9f0381564a4abbdbd7aaf461ca33', 'request_id': 'req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:15.625500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6cc9f0381564a4abbdbd7aaf461ca33', 'request_id': 'req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:15.631140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6cc9f0381564a4abbdbd7aaf461ca33', 'request_id': 'req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:15.642982 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] The token cache is being invalidate because project b6cc9f0381564a4abbdbd7aaf461ca33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:15.643270 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:15.643914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:15.645591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1359/2690] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:44:15 2022] DELETE /identity/v3/projects/b6cc9f0381564a4abbdbd7aaf461ca33 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:15.649665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c6910-1d93-4743-a4b1-f000535913f8 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:15.659968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c6910-1d93-4743-a4b1-f000535913f8 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:15.715966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:15.716782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:15.717056 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:15.717339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] PATH_INFO: `/v3/users/e4639fe08506403ab4bdeb7d61fc05eb` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:15.719192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4639fe08506403ab4bdeb7d61fc05eb)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:15.721243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:15.937503 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4639fe08506403ab4bdeb7d61fc05eb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:15.938195 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4639fe08506403ab4bdeb7d61fc05eb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:15.946403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4639fe08506403ab4bdeb7d61fc05eb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:15.946403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4639fe08506403ab4bdeb7d61fc05eb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:15.958629 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4639fe08506403ab4bdeb7d61fc05eb'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:16.033197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1332/2691] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:44:15 2022] DELETE /identity/v3/users/e4639fe08506403ab4bdeb7d61fc05eb => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:16.461503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:16.462998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:16.476327 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f5560-ced7-4934-8bb7-c5b23cb0e4ad admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:16.534543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:16.535329 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:16.535642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:16.535920 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] PATH_INFO: `/v3/projects/f64bd3d377694138b755fd58a6c36564` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:16.537081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64bd3d377694138b755fd58a6c36564)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:16.539288 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:16.627939 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:16.642360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64bd3d377694138b755fd58a6c36564', 'request_id': 'req-ae4877cb-5323-452f-ba54-f1d8623e5437'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:16.707585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64bd3d377694138b755fd58a6c36564', 'request_id': 'req-ae4877cb-5323-452f-ba54-f1d8623e5437'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:16.713105 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64bd3d377694138b755fd58a6c36564', 'request_id': 'req-ae4877cb-5323-452f-ba54-f1d8623e5437'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:16.724722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] The token cache is being invalidate because project f64bd3d377694138b755fd58a6c36564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:16.725121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:16.725765 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:16.727287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1360/2692] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:44:16 2022] DELETE /identity/v3/projects/f64bd3d377694138b755fd58a6c36564 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:16.731283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:16.732747 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:16.746346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5b7e5-dd72-462e-a2a2-5f3bea702eb1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:16.789140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:16.802315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:16.803229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:16.803515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:16.803826 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] PATH_INFO: `/v3/users/ec2f775946ec49888c5fe8e24738b22d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:16.805039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2f775946ec49888c5fe8e24738b22d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:16.806996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:16.998725 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2f775946ec49888c5fe8e24738b22d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:16.999413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2f775946ec49888c5fe8e24738b22d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:17.006128 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2f775946ec49888c5fe8e24738b22d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:17.006748 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2f775946ec49888c5fe8e24738b22d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:17.018759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-f779a720-fca1-49b9-b7f7-a477ee82935d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2f775946ec49888c5fe8e24738b22d'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:17.094140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1333/2693] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:44:16 2022] DELETE /identity/v3/users/ec2f775946ec49888c5fe8e24738b22d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:17.498187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:17.500211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:17.513780 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4877cb-5323-452f-ba54-f1d8623e5437 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:17.572588 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:17.573449 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:17.573732 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:17.574006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] PATH_INFO: `/v3/projects/28d3d6db37794c7d948147c37dd4eb81` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:17.575337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28d3d6db37794c7d948147c37dd4eb81)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:17.577276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:17.656779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28d3d6db37794c7d948147c37dd4eb81', 'request_id': 'req-8b36aae5-5f91-4470-b3f3-e827d19114b5'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:17.735849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28d3d6db37794c7d948147c37dd4eb81', 'request_id': 'req-8b36aae5-5f91-4470-b3f3-e827d19114b5'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:17.741578 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28d3d6db37794c7d948147c37dd4eb81', 'request_id': 'req-8b36aae5-5f91-4470-b3f3-e827d19114b5'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:17.752409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] The token cache is being invalidate because project 28d3d6db37794c7d948147c37dd4eb81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:17.752694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:17.753318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-8b36aae5-5f91-4470-b3f3-e827d19114b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:17.754851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1361/2694] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:44:17 2022] DELETE /identity/v3/projects/28d3d6db37794c7d948147c37dd4eb81 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:17.760870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0b0c2-4507-46fa-9c7e-821d884ce892 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:17.761347 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0b0c2-4507-46fa-9c7e-821d884ce892 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:17.761458 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0b0c2-4507-46fa-9c7e-821d884ce892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:17.779071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-30c0b0c2-4507-46fa-9c7e-821d884ce892 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:17.780431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-30c0b0c2-4507-46fa-9c7e-821d884ce892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:17.838840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1334/2695] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:17.847812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b6ea5-4b11-4e26-a0fd-b84bb27a52ab None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:17.848098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b6ea5-4b11-4e26-a0fd-b84bb27a52ab None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:17.848372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b6ea5-4b11-4e26-a0fd-b84bb27a52ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:17.865871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-979b6ea5-4b11-4e26-a0fd-b84bb27a52ab None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:17.867063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-979b6ea5-4b11-4e26-a0fd-b84bb27a52ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:17.874784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1362/2696] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:44:17.878837 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0b0c2-4507-46fa-9c7e-821d884ce892 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:17.888346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0b0c2-4507-46fa-9c7e-821d884ce892 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:17.893423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a069ac-f4c4-409d-87fd-798def216611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:17.894071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a069ac-f4c4-409d-87fd-798def216611 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:17.894355 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a069ac-f4c4-409d-87fd-798def216611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:17.894634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a069ac-f4c4-409d-87fd-798def216611 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:17.895356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a069ac-f4c4-409d-87fd-798def216611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:44:17.895702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a069ac-f4c4-409d-87fd-798def216611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:17.897288 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a069ac-f4c4-409d-87fd-798def216611 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:17.909171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1335/2697] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:44:17.913136 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b6ea5-4b11-4e26-a0fd-b84bb27a52ab None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:17.922429 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b6ea5-4b11-4e26-a0fd-b84bb27a52ab None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:17.927021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d01d9c-eb53-4004-88f0-371629a96094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:17.927687 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d01d9c-eb53-4004-88f0-371629a96094 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:17.927966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d01d9c-eb53-4004-88f0-371629a96094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:17.928234 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d01d9c-eb53-4004-88f0-371629a96094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:17.928910 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d01d9c-eb53-4004-88f0-371629a96094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:17.930779 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d01d9c-eb53-4004-88f0-371629a96094 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:18.057068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1363/2698] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:17 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:18.061351 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a069ac-f4c4-409d-87fd-798def216611 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:18.071477 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a069ac-f4c4-409d-87fd-798def216611 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:18.119689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:18.128778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3926fc-9a84-4b00-9503-d3b626b0eb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:18.131693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3926fc-9a84-4b00-9503-d3b626b0eb58 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.131693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3926fc-9a84-4b00-9503-d3b626b0eb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.131693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3926fc-9a84-4b00-9503-d3b626b0eb58 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.131693 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3926fc-9a84-4b00-9503-d3b626b0eb58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:18.132831 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3926fc-9a84-4b00-9503-d3b626b0eb58 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:18.237188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1336/2699] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:18 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:18.242050 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d01d9c-eb53-4004-88f0-371629a96094 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:18.255219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d01d9c-eb53-4004-88f0-371629a96094 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:18.260016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79098f02-2906-4576-9918-9d4824c547ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:18.263670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79098f02-2906-4576-9918-9d4824c547ba admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.263670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79098f02-2906-4576-9918-9d4824c547ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.263670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79098f02-2906-4576-9918-9d4824c547ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.263670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79098f02-2906-4576-9918-9d4824c547ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:18.263670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79098f02-2906-4576-9918-9d4824c547ba admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:18.274936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1364/2700] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:18.278677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3926fc-9a84-4b00-9503-d3b626b0eb58 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:18.288880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3926fc-9a84-4b00-9503-d3b626b0eb58 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:18.293697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01b8b8-bec2-4236-82c4-3e647bd7ddbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:18.294646 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01b8b8-bec2-4236-82c4-3e647bd7ddbc admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.294931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01b8b8-bec2-4236-82c4-3e647bd7ddbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.295212 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01b8b8-bec2-4236-82c4-3e647bd7ddbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.295995 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01b8b8-bec2-4236-82c4-3e647bd7ddbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:18.297625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01b8b8-bec2-4236-82c4-3e647bd7ddbc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:18.308460 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1337/2701] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:18.313583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79098f02-2906-4576-9918-9d4824c547ba admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:18.323811 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79098f02-2906-4576-9918-9d4824c547ba admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:18.329248 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc24c3-dff0-4d53-92cd-51daf0f4c4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:18.330117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc24c3-dff0-4d53-92cd-51daf0f4c4f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.330401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc24c3-dff0-4d53-92cd-51daf0f4c4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.330672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc24c3-dff0-4d53-92cd-51daf0f4c4f8 admin admin] PATH_INFO: `/v3/projects/520d945aca5742a9a61b9e9b69be4a25/users/c43f318dea3d493d8284cc69d0aad571/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.332922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:18.345778 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc24c3-dff0-4d53-92cd-51daf0f4c4f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=520d945aca5742a9a61b9e9b69be4a25, user_id=c43f318dea3d493d8284cc69d0aad571, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:18.348483 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc24c3-dff0-4d53-92cd-51daf0f4c4f8 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:18.431659 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1365/2702] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:18 2022] PUT /identity/v3/projects/520d945aca5742a9a61b9e9b69be4a25/users/c43f318dea3d493d8284cc69d0aad571/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:18.443551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d446e-ce0e-46aa-be9b-8bb33a13f5a7 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.443834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d446e-ce0e-46aa-be9b-8bb33a13f5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.444139 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d446e-ce0e-46aa-be9b-8bb33a13f5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.470566 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-778d446e-ce0e-46aa-be9b-8bb33a13f5a7 None None] MFA Rules not processed for user `c43f318dea3d493d8284cc69d0aad571`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:18.471811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-778d446e-ce0e-46aa-be9b-8bb33a13f5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:18.535134 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1338/2703] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:18.539670 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc24c3-dff0-4d53-92cd-51daf0f4c4f8 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:18.600692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc24c3-dff0-4d53-92cd-51daf0f4c4f8 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:18.605339 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4c8af-35b3-456b-b451-96cab15c45d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:18.605991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4c8af-35b3-456b-b451-96cab15c45d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.606276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4c8af-35b3-456b-b451-96cab15c45d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.606554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4c8af-35b3-456b-b451-96cab15c45d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.607236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4c8af-35b3-456b-b451-96cab15c45d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:18.609679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4c8af-35b3-456b-b451-96cab15c45d9 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:18.700740 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1366/2704] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:18 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:18.704720 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d446e-ce0e-46aa-be9b-8bb33a13f5a7 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:18.715708 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d446e-ce0e-46aa-be9b-8bb33a13f5a7 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:18.770582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999d72e-f094-45a8-9126-927d75bc171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:18.771235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999d72e-f094-45a8-9126-927d75bc171e admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.771516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999d72e-f094-45a8-9126-927d75bc171e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.771822 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999d72e-f094-45a8-9126-927d75bc171e admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.772509 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999d72e-f094-45a8-9126-927d75bc171e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:18.774413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999d72e-f094-45a8-9126-927d75bc171e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:18.877949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1339/2705] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:18 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:18.881840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4c8af-35b3-456b-b451-96cab15c45d9 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:18.892615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4c8af-35b3-456b-b451-96cab15c45d9 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:18.897201 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3067d-34a2-4dd8-b0c7-474c799177e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:18.897846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3067d-34a2-4dd8-b0c7-474c799177e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.898125 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3067d-34a2-4dd8-b0c7-474c799177e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.898396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3067d-34a2-4dd8-b0c7-474c799177e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.899112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3067d-34a2-4dd8-b0c7-474c799177e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:18.900722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3067d-34a2-4dd8-b0c7-474c799177e6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:18.911041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1367/2706] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:18.914770 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999d72e-f094-45a8-9126-927d75bc171e admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:18.925186 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999d72e-f094-45a8-9126-927d75bc171e admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:18.929839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c832aba-8244-4a18-a310-b296cf8201cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:18.930502 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c832aba-8244-4a18-a310-b296cf8201cc admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.930788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c832aba-8244-4a18-a310-b296cf8201cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.931065 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c832aba-8244-4a18-a310-b296cf8201cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.931827 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c832aba-8244-4a18-a310-b296cf8201cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:18.933417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c832aba-8244-4a18-a310-b296cf8201cc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:18.943844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1340/2707] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:18.947768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3067d-34a2-4dd8-b0c7-474c799177e6 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:18.958179 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3067d-34a2-4dd8-b0c7-474c799177e6 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:18.962710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212d5c1-58b3-4e94-a6ca-e85a3fd63d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:18.963608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212d5c1-58b3-4e94-a6ca-e85a3fd63d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:18.963890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212d5c1-58b3-4e94-a6ca-e85a3fd63d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:18.964163 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212d5c1-58b3-4e94-a6ca-e85a3fd63d88 admin admin] PATH_INFO: `/v3/projects/63dbb8d19ee74a4ebdda8be6f1f5b10a/users/067e8aaaef094ecea85234440f5c5947/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:18.978147 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212d5c1-58b3-4e94-a6ca-e85a3fd63d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63dbb8d19ee74a4ebdda8be6f1f5b10a, user_id=067e8aaaef094ecea85234440f5c5947, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:18.980809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212d5c1-58b3-4e94-a6ca-e85a3fd63d88 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:19.093967 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1368/2708] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:18 2022] PUT /identity/v3/projects/63dbb8d19ee74a4ebdda8be6f1f5b10a/users/067e8aaaef094ecea85234440f5c5947/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:44:19.105088 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ef69c-3669-402f-91b5-aa7a652b4952 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:19.105415 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ef69c-3669-402f-91b5-aa7a652b4952 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:19.105699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ef69c-3669-402f-91b5-aa7a652b4952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:19.132512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-f10ef69c-3669-402f-91b5-aa7a652b4952 None None] MFA Rules not processed for user `067e8aaaef094ecea85234440f5c5947`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:19.133773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f10ef69c-3669-402f-91b5-aa7a652b4952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:19.162876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:19.198084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1341/2709] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:19.203138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212d5c1-58b3-4e94-a6ca-e85a3fd63d88 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:19.269306 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212d5c1-58b3-4e94-a6ca-e85a3fd63d88 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:19.275452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504621c2-1e7c-4078-a612-bfc5aec019c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:19.276162 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504621c2-1e7c-4078-a612-bfc5aec019c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:19.276446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504621c2-1e7c-4078-a612-bfc5aec019c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:19.276722 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504621c2-1e7c-4078-a612-bfc5aec019c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:19.277409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504621c2-1e7c-4078-a612-bfc5aec019c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:19.279322 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504621c2-1e7c-4078-a612-bfc5aec019c8 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:19.362845 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1369/2710] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:19 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:19.364778 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ef69c-3669-402f-91b5-aa7a652b4952 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:19.376380 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ef69c-3669-402f-91b5-aa7a652b4952 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:19.431886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513b20b-d718-48fe-aebe-93e1400064a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:19.432487 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513b20b-d718-48fe-aebe-93e1400064a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:19.432774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513b20b-d718-48fe-aebe-93e1400064a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:19.433053 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513b20b-d718-48fe-aebe-93e1400064a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:19.433740 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513b20b-d718-48fe-aebe-93e1400064a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:19.435706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513b20b-d718-48fe-aebe-93e1400064a8 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:19.541769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1342/2711] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:19 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:44:19.546678 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504621c2-1e7c-4078-a612-bfc5aec019c8 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:19.557185 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504621c2-1e7c-4078-a612-bfc5aec019c8 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:19.562784 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e30a24-ccc7-48a0-83a5-541c7f2eab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:19.563451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e30a24-ccc7-48a0-83a5-541c7f2eab26 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:19.563788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e30a24-ccc7-48a0-83a5-541c7f2eab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:19.564063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e30a24-ccc7-48a0-83a5-541c7f2eab26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:19.564782 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e30a24-ccc7-48a0-83a5-541c7f2eab26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:19.566376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e30a24-ccc7-48a0-83a5-541c7f2eab26 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:19.577227 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1370/2712] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:19.581521 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513b20b-d718-48fe-aebe-93e1400064a8 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:19.591499 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513b20b-d718-48fe-aebe-93e1400064a8 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:19.596656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f76ea-e953-4a84-b62e-c3e91149e980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:19.597537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f76ea-e953-4a84-b62e-c3e91149e980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:19.597824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f76ea-e953-4a84-b62e-c3e91149e980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:19.598104 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f76ea-e953-4a84-b62e-c3e91149e980 admin admin] PATH_INFO: `/v3/projects/a782b1b9d15a4d019db0341f33877767/users/8578b5c29be7493eb66636376e5e0665/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:19.613199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f76ea-e953-4a84-b62e-c3e91149e980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a782b1b9d15a4d019db0341f33877767, user_id=8578b5c29be7493eb66636376e5e0665, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:19.616033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f76ea-e953-4a84-b62e-c3e91149e980 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:19.693991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1343/2713] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:19 2022] PUT /identity/v3/projects/a782b1b9d15a4d019db0341f33877767/users/8578b5c29be7493eb66636376e5e0665/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:19.698117 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e30a24-ccc7-48a0-83a5-541c7f2eab26 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:19.708363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e30a24-ccc7-48a0-83a5-541c7f2eab26 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:19.712059 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:19.764573 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb197f14-defc-4d2d-9341-aee6ecd77848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:19.765265 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb197f14-defc-4d2d-9341-aee6ecd77848 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:19.765548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb197f14-defc-4d2d-9341-aee6ecd77848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:19.765827 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb197f14-defc-4d2d-9341-aee6ecd77848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:19.766548 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb197f14-defc-4d2d-9341-aee6ecd77848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:19.768229 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb197f14-defc-4d2d-9341-aee6ecd77848 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:19.778261 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1371/2714] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:19.782885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f76ea-e953-4a84-b62e-c3e91149e980 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:19.793082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f76ea-e953-4a84-b62e-c3e91149e980 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:19.798562 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8ce28-f631-4b56-b1a4-714baf7bad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:19.799429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8ce28-f631-4b56-b1a4-714baf7bad28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:19.799742 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8ce28-f631-4b56-b1a4-714baf7bad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:19.800014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8ce28-f631-4b56-b1a4-714baf7bad28 admin admin] PATH_INFO: `/v3/projects/a782b1b9d15a4d019db0341f33877767/users/8578b5c29be7493eb66636376e5e0665/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:19.801980 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8ce28-f631-4b56-b1a4-714baf7bad28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a782b1b9d15a4d019db0341f33877767, user_id=8578b5c29be7493eb66636376e5e0665, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:19.804634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8ce28-f631-4b56-b1a4-714baf7bad28 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:19.814452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1344/2715] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:19 2022] PUT /identity/v3/projects/a782b1b9d15a4d019db0341f33877767/users/8578b5c29be7493eb66636376e5e0665/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:19.824186 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bcc53-f700-4285-8649-cd399411842c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:19.824479 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bcc53-f700-4285-8649-cd399411842c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:19.824756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bcc53-f700-4285-8649-cd399411842c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:19.850718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-d59bcc53-f700-4285-8649-cd399411842c None None] MFA Rules not processed for user `8578b5c29be7493eb66636376e5e0665`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:19.851964 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d59bcc53-f700-4285-8649-cd399411842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:19.918929 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1372/2716] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:44:19.931432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8ce28-f631-4b56-b1a4-714baf7bad28 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:19.933188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e2d8ce28-f631-4b56-b1a4-714baf7bad28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:19.992110 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8ce28-f631-4b56-b1a4-714baf7bad28 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:19.995825 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b7a4d-e66e-4acb-ad78-97e8e7c1a630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:19.996473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b7a4d-e66e-4acb-ad78-97e8e7c1a630 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:19.996759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b7a4d-e66e-4acb-ad78-97e8e7c1a630 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:19.997037 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b7a4d-e66e-4acb-ad78-97e8e7c1a630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:20.031094 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:20.053273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b7a4d-e66e-4acb-ad78-97e8e7c1a630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:20.054713 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b7a4d-e66e-4acb-ad78-97e8e7c1a630 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:20.063626 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1345/2717] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:21.942998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bcc53-f700-4285-8649-cd399411842c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:21.944363 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-d59bcc53-f700-4285-8649-cd399411842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:21.956216 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bcc53-f700-4285-8649-cd399411842c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:21.959852 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fc89b-b792-4907-b336-1af3b97f4260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:21.960487 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fc89b-b792-4907-b336-1af3b97f4260 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:21.960772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fc89b-b792-4907-b336-1af3b97f4260 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:21.961102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fc89b-b792-4907-b336-1af3b97f4260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:22.016396 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fc89b-b792-4907-b336-1af3b97f4260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:22.017823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3fc89b-b792-4907-b336-1af3b97f4260 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:22.027068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1373/2718] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:23.313279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b7a4d-e66e-4acb-ad78-97e8e7c1a630 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:23.322122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b7a4d-e66e-4acb-ad78-97e8e7c1a630 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:23.325801 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c0420-608a-42c8-a991-c24bf97f26e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:23.326438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c0420-608a-42c8-a991-c24bf97f26e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:23.326723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c0420-608a-42c8-a991-c24bf97f26e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:23.327003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c0420-608a-42c8-a991-c24bf97f26e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:23.336793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c0420-608a-42c8-a991-c24bf97f26e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:23.338280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c0420-608a-42c8-a991-c24bf97f26e2 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:23.347953 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1346/2719] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:27.872320 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fc89b-b792-4907-b336-1af3b97f4260 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:27.901812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=100 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:27.933100 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fc89b-b792-4907-b336-1af3b97f4260 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:27.938822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:27.939731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:27.940016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:27.940291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] PATH_INFO: `/v3/users/c43f318dea3d493d8284cc69d0aad571` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:27.941522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c43f318dea3d493d8284cc69d0aad571)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:27.943646 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:28.138083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c43f318dea3d493d8284cc69d0aad571'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:28.231180 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c43f318dea3d493d8284cc69d0aad571'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:28.231831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c43f318dea3d493d8284cc69d0aad571'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:28.239843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c43f318dea3d493d8284cc69d0aad571'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:28.240556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c43f318dea3d493d8284cc69d0aad571'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:28.258473 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1374/2720] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:44:27 2022] DELETE /identity/v3/users/c43f318dea3d493d8284cc69d0aad571 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:28.262976 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3254f0-34ee-4c59-9074-6e391281b9df None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:28.263480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3254f0-34ee-4c59-9074-6e391281b9df None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:28.263619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3254f0-34ee-4c59-9074-6e391281b9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:28.280265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7c3254f0-34ee-4c59-9074-6e391281b9df None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:28.281473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7c3254f0-34ee-4c59-9074-6e391281b9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:28.340080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1347/2721] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:44:28.349562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:28.350991 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:28.406880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d3d5b-a9cb-4de3-b5ca-9d217a93d5d5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:28.410413 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ca362-7828-4d13-85f7-fb91f649f4d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:28.411046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ca362-7828-4d13-85f7-fb91f649f4d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:28.411326 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ca362-7828-4d13-85f7-fb91f649f4d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:28.411633 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ca362-7828-4d13-85f7-fb91f649f4d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:28.421107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ca362-7828-4d13-85f7-fb91f649f4d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:28.423046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ca362-7828-4d13-85f7-fb91f649f4d6 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:28.432350 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1375/2722] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:28.822989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3254f0-34ee-4c59-9074-6e391281b9df None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:28.825488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7c3254f0-34ee-4c59-9074-6e391281b9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:28.836480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3254f0-34ee-4c59-9074-6e391281b9df None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:28.841550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:28.842398 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:28.842677 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:28.842952 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] PATH_INFO: `/v3/projects/520d945aca5742a9a61b9e9b69be4a25` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:28.844798 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=520d945aca5742a9a61b9e9b69be4a25)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:28.846709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:28.947773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '520d945aca5742a9a61b9e9b69be4a25', 'request_id': 'req-389906f7-908d-411b-8c15-97016699c415'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:28.953255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '520d945aca5742a9a61b9e9b69be4a25', 'request_id': 'req-389906f7-908d-411b-8c15-97016699c415'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:28.959961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '520d945aca5742a9a61b9e9b69be4a25', 'request_id': 'req-389906f7-908d-411b-8c15-97016699c415'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.030397 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] The token cache is being invalidate because project 520d945aca5742a9a61b9e9b69be4a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:29.030699 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.031368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.033673 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1348/2723] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:44:28 2022] DELETE /identity/v3/projects/520d945aca5742a9a61b9e9b69be4a25 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:29.037762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ca362-7828-4d13-85f7-fb91f649f4d6 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:29.047901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ca362-7828-4d13-85f7-fb91f649f4d6 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:29.091787 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:29.103860 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:29.104785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:29.105178 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:29.105456 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] PATH_INFO: `/v3/users/067e8aaaef094ecea85234440f5c5947` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:29.106688 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=067e8aaaef094ecea85234440f5c5947)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:29.108660 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:29.282533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '067e8aaaef094ecea85234440f5c5947'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.358813 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '067e8aaaef094ecea85234440f5c5947'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.359428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '067e8aaaef094ecea85234440f5c5947'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.366780 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '067e8aaaef094ecea85234440f5c5947'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.367406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '067e8aaaef094ecea85234440f5c5947'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.385053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1376/2724] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:44:29 2022] DELETE /identity/v3/users/067e8aaaef094ecea85234440f5c5947 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:29.804040 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:29.805557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:29.820620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389906f7-908d-411b-8c15-97016699c415 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:29.865930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:29.878116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:29.878978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:29.879259 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:29.879555 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] PATH_INFO: `/v3/projects/63dbb8d19ee74a4ebdda8be6f1f5b10a` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:29.880738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63dbb8d19ee74a4ebdda8be6f1f5b10a)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:29.882643 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:29.981738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63dbb8d19ee74a4ebdda8be6f1f5b10a', 'request_id': 'req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.987272 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63dbb8d19ee74a4ebdda8be6f1f5b10a', 'request_id': 'req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:29.993560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63dbb8d19ee74a4ebdda8be6f1f5b10a', 'request_id': 'req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:30.067621 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] The token cache is being invalidate because project 63dbb8d19ee74a4ebdda8be6f1f5b10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:30.067848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:30.068472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:30.070047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1349/2725] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:44:29 2022] DELETE /identity/v3/projects/63dbb8d19ee74a4ebdda8be6f1f5b10a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:44:30.074019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:30.075450 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:30.088731 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414fb62-f535-4961-afb5-6665d451e80a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:30.144315 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:30.145183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:30.145465 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:30.145741 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] PATH_INFO: `/v3/users/8578b5c29be7493eb66636376e5e0665` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:30.146954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8578b5c29be7493eb66636376e5e0665)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:30.148905 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:30.349682 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8578b5c29be7493eb66636376e5e0665'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:30.432890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8578b5c29be7493eb66636376e5e0665'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:30.433563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8578b5c29be7493eb66636376e5e0665'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:30.441174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8578b5c29be7493eb66636376e5e0665'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:30.441812 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-3fb71e59-daf0-4a96-9a9a-4b3ac4b100df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8578b5c29be7493eb66636376e5e0665'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:30.447219 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:30.459769 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1377/2726] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:44:30 2022] DELETE /identity/v3/users/8578b5c29be7493eb66636376e5e0665 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:30.852349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:30.853824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:30.867033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eadad4-cb9d-4154-afe6-ab0517eb5ce1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:30.926192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:30.927064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:30.927352 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:30.927665 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] PATH_INFO: `/v3/projects/a782b1b9d15a4d019db0341f33877767` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:30.928862 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a782b1b9d15a4d019db0341f33877767)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:30.930776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:31.056875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a782b1b9d15a4d019db0341f33877767', 'request_id': 'req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:31.062416 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a782b1b9d15a4d019db0341f33877767', 'request_id': 'req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:31.068611 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a782b1b9d15a4d019db0341f33877767', 'request_id': 'req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:31.149429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] The token cache is being invalidate because project a782b1b9d15a4d019db0341f33877767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:31.149934 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:31.150559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-42c1b82f-8b3d-470c-bb2a-cf8958a4c6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:31.152153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1350/2727] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:44:30 2022] DELETE /identity/v3/projects/a782b1b9d15a4d019db0341f33877767 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:31.157539 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124375e3-5993-4309-8236-ac647721171e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.158068 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124375e3-5993-4309-8236-ac647721171e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.158187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124375e3-5993-4309-8236-ac647721171e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.176869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-124375e3-5993-4309-8236-ac647721171e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:31.182685 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-124375e3-5993-4309-8236-ac647721171e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:31.236222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1378/2728] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:31.244842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b002a-7942-4cab-9d6f-592d0c60ace7 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.245116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b002a-7942-4cab-9d6f-592d0c60ace7 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.245384 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b002a-7942-4cab-9d6f-592d0c60ace7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.257715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:31.263657 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-1f7b002a-7942-4cab-9d6f-592d0c60ace7 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:31.270262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-1f7b002a-7942-4cab-9d6f-592d0c60ace7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:31.273238 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1351/2729] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:31.277775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124375e3-5993-4309-8236-ac647721171e None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:31.289353 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124375e3-5993-4309-8236-ac647721171e None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:31.295186 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc66f78-b143-4148-a43e-0957158cfc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:31.302864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc66f78-b143-4148-a43e-0957158cfc55 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.302864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc66f78-b143-4148-a43e-0957158cfc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.302864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc66f78-b143-4148-a43e-0957158cfc55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.302864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc66f78-b143-4148-a43e-0957158cfc55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:44:31.302864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc66f78-b143-4148-a43e-0957158cfc55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:31.302864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc66f78-b143-4148-a43e-0957158cfc55 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:31.311018 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1379/2730] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:44:31.315654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b002a-7942-4cab-9d6f-592d0c60ace7 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:31.324841 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b002a-7942-4cab-9d6f-592d0c60ace7 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:31.330273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c232cd-b0f1-49ea-b6a1-104855633c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:31.331519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c232cd-b0f1-49ea-b6a1-104855633c9b admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.331519 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c232cd-b0f1-49ea-b6a1-104855633c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.331770 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c232cd-b0f1-49ea-b6a1-104855633c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.332351 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c232cd-b0f1-49ea-b6a1-104855633c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:31.334248 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c232cd-b0f1-49ea-b6a1-104855633c9b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:31.404383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1352/2731] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:31 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:31.409280 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc66f78-b143-4148-a43e-0957158cfc55 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:31.419996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc66f78-b143-4148-a43e-0957158cfc55 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:31.476064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e7c9a-bbd3-4097-b756-7e0a870f61cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:31.476733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e7c9a-bbd3-4097-b756-7e0a870f61cf admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.477018 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e7c9a-bbd3-4097-b756-7e0a870f61cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.477298 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e7c9a-bbd3-4097-b756-7e0a870f61cf admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.478019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e7c9a-bbd3-4097-b756-7e0a870f61cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:31.479968 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e7c9a-bbd3-4097-b756-7e0a870f61cf admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:31.557004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1380/2732] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:31 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:44:31.561180 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c232cd-b0f1-49ea-b6a1-104855633c9b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:31.572005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c232cd-b0f1-49ea-b6a1-104855633c9b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:31.576755 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989201a-a7f1-43b3-b903-295b5064a434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:31.577411 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989201a-a7f1-43b3-b903-295b5064a434 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.577821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989201a-a7f1-43b3-b903-295b5064a434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.578188 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989201a-a7f1-43b3-b903-295b5064a434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.579589 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989201a-a7f1-43b3-b903-295b5064a434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:31.581243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989201a-a7f1-43b3-b903-295b5064a434 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:31.591426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1353/2733] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:31.595115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e7c9a-bbd3-4097-b756-7e0a870f61cf admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:31.605213 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e7c9a-bbd3-4097-b756-7e0a870f61cf admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:31.609970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13b0a8-51df-4b0a-9ac4-78cdf73f7ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:31.610608 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13b0a8-51df-4b0a-9ac4-78cdf73f7ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.610895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13b0a8-51df-4b0a-9ac4-78cdf73f7ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.613901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13b0a8-51df-4b0a-9ac4-78cdf73f7ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.613901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13b0a8-51df-4b0a-9ac4-78cdf73f7ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:31.613901 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13b0a8-51df-4b0a-9ac4-78cdf73f7ce6 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:31.626409 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1381/2734] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:31.630565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989201a-a7f1-43b3-b903-295b5064a434 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:31.640475 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989201a-a7f1-43b3-b903-295b5064a434 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:31.645845 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eafd29-cbb5-4c18-b077-8309917bc20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:31.646787 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eafd29-cbb5-4c18-b077-8309917bc20a admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.647005 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eafd29-cbb5-4c18-b077-8309917bc20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.647290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eafd29-cbb5-4c18-b077-8309917bc20a admin admin] PATH_INFO: `/v3/projects/07087a801c83417aae147803fc643ff7/users/5b1c00907c394971bb14f755b25b1d07/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.661754 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eafd29-cbb5-4c18-b077-8309917bc20a admin admin] RBAC: Authorizing `identity:create_grant(project_id=07087a801c83417aae147803fc643ff7, user_id=5b1c00907c394971bb14f755b25b1d07, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:31.664445 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eafd29-cbb5-4c18-b077-8309917bc20a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:31.759417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1354/2735] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:31 2022] PUT /identity/v3/projects/07087a801c83417aae147803fc643ff7/users/5b1c00907c394971bb14f755b25b1d07/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:31.822096 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8e19-8f64-4b7f-b755-f1e2ecc2b9e9 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.822388 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8e19-8f64-4b7f-b755-f1e2ecc2b9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.822668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf8e19-8f64-4b7f-b755-f1e2ecc2b9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.848926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-9fdf8e19-8f64-4b7f-b755-f1e2ecc2b9e9 None None] MFA Rules not processed for user `5b1c00907c394971bb14f755b25b1d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:31.850148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9fdf8e19-8f64-4b7f-b755-f1e2ecc2b9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:31.893246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:31.915636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1382/2736] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:31.926902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eafd29-cbb5-4c18-b077-8309917bc20a admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:31.928667 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-75eafd29-cbb5-4c18-b077-8309917bc20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:31.986171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eafd29-cbb5-4c18-b077-8309917bc20a admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:31.989824 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0bc352-848a-4f1b-905d-9eb4d42badf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:31.990459 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bc352-848a-4f1b-905d-9eb4d42badf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:31.990740 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bc352-848a-4f1b-905d-9eb4d42badf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:31.991014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bc352-848a-4f1b-905d-9eb4d42badf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:31.999214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0bc352-848a-4f1b-905d-9eb4d42badf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:32.001373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0bc352-848a-4f1b-905d-9eb4d42badf9 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:32.010170 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1355/2737] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:34.979041 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf8e19-8f64-4b7f-b755-f1e2ecc2b9e9 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:35.039541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf8e19-8f64-4b7f-b755-f1e2ecc2b9e9 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:35.044376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:35.045253 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:35.045535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:35.045810 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] PATH_INFO: `/v3/users/5b1c00907c394971bb14f755b25b1d07` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:35.047007 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1c00907c394971bb14f755b25b1d07)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:35.049435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:35.263873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1c00907c394971bb14f755b25b1d07'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:35.336013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1c00907c394971bb14f755b25b1d07'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:35.336783 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1c00907c394971bb14f755b25b1d07'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:35.345477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1c00907c394971bb14f755b25b1d07'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:35.346138 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1c00907c394971bb14f755b25b1d07'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:35.363691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1383/2738] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:44:34 2022] DELETE /identity/v3/users/5b1c00907c394971bb14f755b25b1d07 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:35.368292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97660172-ac7b-4991-a396-4b5da34a176f None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:35.368766 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97660172-ac7b-4991-a396-4b5da34a176f None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:35.368877 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97660172-ac7b-4991-a396-4b5da34a176f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:35.385424 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-97660172-ac7b-4991-a396-4b5da34a176f None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:35.386638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-97660172-ac7b-4991-a396-4b5da34a176f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:35.446425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1356/2739] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:35.456134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:35.457585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:35.514090 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188b175-b7a3-488d-896c-d10f74538b56 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:35.517647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04133a13-04cf-4eff-8197-a6e323efd5fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:35.518284 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04133a13-04cf-4eff-8197-a6e323efd5fc service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:35.518568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04133a13-04cf-4eff-8197-a6e323efd5fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:35.518850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04133a13-04cf-4eff-8197-a6e323efd5fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:35.527925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04133a13-04cf-4eff-8197-a6e323efd5fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:35.529358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04133a13-04cf-4eff-8197-a6e323efd5fc service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:35.539467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1384/2740] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:35.966325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97660172-ac7b-4991-a396-4b5da34a176f None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:35.967962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-97660172-ac7b-4991-a396-4b5da34a176f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:35.983431 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97660172-ac7b-4991-a396-4b5da34a176f None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:35.988872 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:35.991684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:35.991684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:35.991684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] PATH_INFO: `/v3/projects/07087a801c83417aae147803fc643ff7` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:35.991684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07087a801c83417aae147803fc643ff7)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:35.993593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:36.102772 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:36.109231 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07087a801c83417aae147803fc643ff7', 'request_id': 'req-dc7eb06d-b33d-4f9a-97f1-bce417918516'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:36.114605 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07087a801c83417aae147803fc643ff7', 'request_id': 'req-dc7eb06d-b33d-4f9a-97f1-bce417918516'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:36.120783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07087a801c83417aae147803fc643ff7', 'request_id': 'req-dc7eb06d-b33d-4f9a-97f1-bce417918516'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:36.184813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] The token cache is being invalidate because project 07087a801c83417aae147803fc643ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:44:36.184991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:36.185746 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:44:36.196541 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1357/2741] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:44:35 2022] DELETE /identity/v3/projects/07087a801c83417aae147803fc643ff7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:36.197135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ca747-f01f-4838-936d-d697a2d7ae8e None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.197135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ca747-f01f-4838-936d-d697a2d7ae8e None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.197135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ca747-f01f-4838-936d-d697a2d7ae8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.201242 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:36.202816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:36.204330 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:36.214454 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-cc0ca747-f01f-4838-936d-d697a2d7ae8e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:36.215675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-cc0ca747-f01f-4838-936d-d697a2d7ae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:36.262705 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7eb06d-b33d-4f9a-97f1-bce417918516 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:36.267054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169d59f-2daf-46d7-bb2d-6f2e25da66c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:36.267759 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169d59f-2daf-46d7-bb2d-6f2e25da66c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.268047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169d59f-2daf-46d7-bb2d-6f2e25da66c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.268326 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169d59f-2daf-46d7-bb2d-6f2e25da66c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.274033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-2169d59f-2daf-46d7-bb2d-6f2e25da66c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:36.278756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1385/2742] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:36.286428 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe76c05-b7fa-47a5-bdfd-cf36c64143f3 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.286716 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe76c05-b7fa-47a5-bdfd-cf36c64143f3 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.287000 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe76c05-b7fa-47a5-bdfd-cf36c64143f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.305013 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-2fe76c05-b7fa-47a5-bdfd-cf36c64143f3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:36.306273 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2fe76c05-b7fa-47a5-bdfd-cf36c64143f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:36.315076 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1386/2743] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:44:36.319140 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe76c05-b7fa-47a5-bdfd-cf36c64143f3 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:36.328362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe76c05-b7fa-47a5-bdfd-cf36c64143f3 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:36.333679 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c988ddd-f17d-4df8-bd86-3c4027f51d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:36.334317 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c988ddd-f17d-4df8-bd86-3c4027f51d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.334593 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c988ddd-f17d-4df8-bd86-3c4027f51d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.334867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c988ddd-f17d-4df8-bd86-3c4027f51d12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.335610 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c988ddd-f17d-4df8-bd86-3c4027f51d12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:44:36.335917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c988ddd-f17d-4df8-bd86-3c4027f51d12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:36.337495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c988ddd-f17d-4df8-bd86-3c4027f51d12 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:36.341010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169d59f-2daf-46d7-bb2d-6f2e25da66c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:36.342684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169d59f-2daf-46d7-bb2d-6f2e25da66c6 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:36.348987 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1387/2744] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:44:36.353451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c988ddd-f17d-4df8-bd86-3c4027f51d12 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:36.355514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1358/2745] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:36.364347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c988ddd-f17d-4df8-bd86-3c4027f51d12 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:36.369081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201d473-5dda-438e-a3bf-64395f52fa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:36.369728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201d473-5dda-438e-a3bf-64395f52fa91 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.370012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201d473-5dda-438e-a3bf-64395f52fa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.370290 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201d473-5dda-438e-a3bf-64395f52fa91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.371115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201d473-5dda-438e-a3bf-64395f52fa91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:36.372895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201d473-5dda-438e-a3bf-64395f52fa91 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:36.480338 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1388/2746] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:44:36 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:44:36.484423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169d59f-2daf-46d7-bb2d-6f2e25da66c6 service cinder] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:36.495875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169d59f-2daf-46d7-bb2d-6f2e25da66c6 service cinder] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:36.551656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d08f9a-b5ac-46ae-9828-447026af9e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:36.552376 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d08f9a-b5ac-46ae-9828-447026af9e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.552647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d08f9a-b5ac-46ae-9828-447026af9e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.552925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d08f9a-b5ac-46ae-9828-447026af9e45 admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.553619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d08f9a-b5ac-46ae-9828-447026af9e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:36.555577 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d08f9a-b5ac-46ae-9828-447026af9e45 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:36.655975 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1359/2747] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:44:36 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:44:36.660014 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201d473-5dda-438e-a3bf-64395f52fa91 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:36.670581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201d473-5dda-438e-a3bf-64395f52fa91 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:36.675848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc0c20-61a7-4bf4-a512-c1555b538432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:36.676500 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc0c20-61a7-4bf4-a512-c1555b538432 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.676785 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc0c20-61a7-4bf4-a512-c1555b538432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.677064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc0c20-61a7-4bf4-a512-c1555b538432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.677840 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc0c20-61a7-4bf4-a512-c1555b538432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:36.679977 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc0c20-61a7-4bf4-a512-c1555b538432 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:36.690948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1389/2748] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:36.694876 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d08f9a-b5ac-46ae-9828-447026af9e45 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:36.704971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d08f9a-b5ac-46ae-9828-447026af9e45 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:36.709650 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359640c-a090-414c-8123-94bc6588c476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:36.710299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359640c-a090-414c-8123-94bc6588c476 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.710588 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359640c-a090-414c-8123-94bc6588c476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.710869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359640c-a090-414c-8123-94bc6588c476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.711628 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359640c-a090-414c-8123-94bc6588c476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:36.713249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359640c-a090-414c-8123-94bc6588c476 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:36.723183 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1360/2749] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:44:36.727081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc0c20-61a7-4bf4-a512-c1555b538432 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:36.737381 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc0c20-61a7-4bf4-a512-c1555b538432 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:36.742016 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95359e0a-79ac-44ab-ad01-57acbacd5577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:36.742883 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95359e0a-79ac-44ab-ad01-57acbacd5577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.743164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95359e0a-79ac-44ab-ad01-57acbacd5577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.743444 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95359e0a-79ac-44ab-ad01-57acbacd5577 admin admin] PATH_INFO: `/v3/projects/428f2755baac4091a9d78e86090d819b/users/b432b13ffa7d42379fe38d6eed54e75f/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.758360 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95359e0a-79ac-44ab-ad01-57acbacd5577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428f2755baac4091a9d78e86090d819b, user_id=b432b13ffa7d42379fe38d6eed54e75f, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:36.761053 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95359e0a-79ac-44ab-ad01-57acbacd5577 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:36.861913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1390/2750] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:44:36 2022] PUT /identity/v3/projects/428f2755baac4091a9d78e86090d819b/users/b432b13ffa7d42379fe38d6eed54e75f/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:44:36.867073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172f722-f51b-40d5-bb67-c499266dc9e3 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:36.867388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172f722-f51b-40d5-bb67-c499266dc9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:36.867700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172f722-f51b-40d5-bb67-c499266dc9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:36.873551 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:36.885580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-8172f722-f51b-40d5-bb67-c499266dc9e3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:36.886765 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-8172f722-f51b-40d5-bb67-c499266dc9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:36.946575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1361/2751] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:44:36.955676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95359e0a-79ac-44ab-ad01-57acbacd5577 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:36.957434 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-95359e0a-79ac-44ab-ad01-57acbacd5577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:37.013274 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95359e0a-79ac-44ab-ad01-57acbacd5577 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:37.017022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfe210-cdb8-4cfa-85d1-c555623a0060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:37.017652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfe210-cdb8-4cfa-85d1-c555623a0060 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:37.017932 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfe210-cdb8-4cfa-85d1-c555623a0060 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:37.018203 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfe210-cdb8-4cfa-85d1-c555623a0060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:37.027134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfe210-cdb8-4cfa-85d1-c555623a0060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:37.028585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfe210-cdb8-4cfa-85d1-c555623a0060 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:37.038347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1391/2752] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:44:38.143337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6f918-9dc0-4d3d-9204-2cad5ffb3c74 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:38.143914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6f918-9dc0-4d3d-9204-2cad5ffb3c74 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:38.144021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6f918-9dc0-4d3d-9204-2cad5ffb3c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:38.171417 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-91d6f918-9dc0-4d3d-9204-2cad5ffb3c74 None None] MFA Rules not processed for user `b432b13ffa7d42379fe38d6eed54e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:38.173205 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-91d6f918-9dc0-4d3d-9204-2cad5ffb3c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:38.241095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1362/2753] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:38.252311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfe210-cdb8-4cfa-85d1-c555623a0060 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:38.291422 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:38.314139 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfe210-cdb8-4cfa-85d1-c555623a0060 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:38.318336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88032450-89a1-4d9e-9353-ed01ccab7fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:38.318978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88032450-89a1-4d9e-9353-ed01ccab7fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:38.319262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88032450-89a1-4d9e-9353-ed01ccab7fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:38.319557 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88032450-89a1-4d9e-9353-ed01ccab7fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:38.328073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88032450-89a1-4d9e-9353-ed01ccab7fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:38.329645 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88032450-89a1-4d9e-9353-ed01ccab7fd6 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:38.338675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1392/2754] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:40.019553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e22e-3a02-4f9e-bcf9-2d20bbe6731e None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:40.020069 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e22e-3a02-4f9e-bcf9-2d20bbe6731e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:40.020190 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e22e-3a02-4f9e-bcf9-2d20bbe6731e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:40.037944 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-41a5e22e-3a02-4f9e-bcf9-2d20bbe6731e None None] MFA Rules not processed for user `b432b13ffa7d42379fe38d6eed54e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:40.039143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-41a5e22e-3a02-4f9e-bcf9-2d20bbe6731e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:40.046185 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1363/2755] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:40.526131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88032450-89a1-4d9e-9353-ed01ccab7fd6 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:40.537199 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88032450-89a1-4d9e-9353-ed01ccab7fd6 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:40.540844 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf159d-0f1a-471c-964d-55847805a577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:40.541477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf159d-0f1a-471c-964d-55847805a577 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:40.541756 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf159d-0f1a-471c-964d-55847805a577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:40.542022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf159d-0f1a-471c-964d-55847805a577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:40.544764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2abf159d-0f1a-471c-964d-55847805a577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:40.554448 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf159d-0f1a-471c-964d-55847805a577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:40.555914 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf159d-0f1a-471c-964d-55847805a577 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:40.565985 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1393/2756] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:42.378436 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e22e-3a02-4f9e-bcf9-2d20bbe6731e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:44:42.379869 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-41a5e22e-3a02-4f9e-bcf9-2d20bbe6731e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:42.448426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e22e-3a02-4f9e-bcf9-2d20bbe6731e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:44:42.453899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24e460-f2f9-4226-87ac-51d22e3ba4ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:44:42.454537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24e460-f2f9-4226-87ac-51d22e3ba4ca service placement] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:42.454813 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24e460-f2f9-4226-87ac-51d22e3ba4ca service placement] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:42.455084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24e460-f2f9-4226-87ac-51d22e3ba4ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:42.456902 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-bb24e460-f2f9-4226-87ac-51d22e3ba4ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:42.468281 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24e460-f2f9-4226-87ac-51d22e3ba4ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:44:42.470508 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24e460-f2f9-4226-87ac-51d22e3ba4ca service placement] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:44:42.480182 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1364/2757] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:44:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:44:44.431027 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9c46-067b-4efa-b834-aea23a4da821 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:44:44.431569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9c46-067b-4efa-b834-aea23a4da821 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:44:44.431705 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc9c46-067b-4efa-b834-aea23a4da821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:44:44.466543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c0cc9c46-067b-4efa-b834-aea23a4da821 None None] MFA Rules not processed for user `2720d45854ab4a61b3b22dd43a523d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:44:44.466543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c0cc9c46-067b-4efa-b834-aea23a4da821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:44:44.519635 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1394/2758] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:44:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:44:52.476095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=93 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:44:54.511442 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=43 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:45:51.404035 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d899c-dff0-429d-9168-10ef783257b4 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:45:51.404535 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d899c-dff0-429d-9168-10ef783257b4 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:45:51.404649 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d899c-dff0-429d-9168-10ef783257b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:45:51.422803 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-038d899c-dff0-429d-9168-10ef783257b4 None None] MFA Rules not processed for user `b432b13ffa7d42379fe38d6eed54e75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:45:51.424014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-038d899c-dff0-429d-9168-10ef783257b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:45:51.430930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1365/2759] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:45:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:46:01.417333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:46:01.670617 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc9c46-067b-4efa-b834-aea23a4da821 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:01.672030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c0cc9c46-067b-4efa-b834-aea23a4da821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:01.684838 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc9c46-067b-4efa-b834-aea23a4da821 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:01.688430 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17395477-2c88-45ee-aa0e-096d6db206d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:01.689066 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17395477-2c88-45ee-aa0e-096d6db206d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:01.689379 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17395477-2c88-45ee-aa0e-096d6db206d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:01.689656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17395477-2c88-45ee-aa0e-096d6db206d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:01.691390 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-17395477-2c88-45ee-aa0e-096d6db206d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:01.760846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17395477-2c88-45ee-aa0e-096d6db206d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:01.762252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17395477-2c88-45ee-aa0e-096d6db206d8 service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:01.774030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1395/2760] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:46:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:46:04.790235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d899c-dff0-429d-9168-10ef783257b4 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:04.791658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-038d899c-dff0-429d-9168-10ef783257b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:04.803782 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d899c-dff0-429d-9168-10ef783257b4 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:04.807949 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:04.808617 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:04.808899 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:04.809175 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:04.810880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:04.824664 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:04.826081 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:04.837125 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1366/2761] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:46:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:46:11.768208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=34 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:46:14.832169 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=6 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:46:52.101272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17395477-2c88-45ee-aa0e-096d6db206d8 service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:52.111309 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17395477-2c88-45ee-aa0e-096d6db206d8 service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:52.115256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05aa3c-bca5-4e58-841d-a6ef7463b385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:52.116006 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05aa3c-bca5-4e58-841d-a6ef7463b385 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:52.116460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05aa3c-bca5-4e58-841d-a6ef7463b385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:52.116638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05aa3c-bca5-4e58-841d-a6ef7463b385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:52.118343 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7e05aa3c-bca5-4e58-841d-a6ef7463b385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:52.128770 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05aa3c-bca5-4e58-841d-a6ef7463b385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:52.130191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05aa3c-bca5-4e58-841d-a6ef7463b385 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:52.139246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1396/2762] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:46:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:46:53.122066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:53.123635 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:53.138329 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b8ff2-3b00-4462-b4db-19d024330be2 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:53.144012 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec83bc-c466-47b6-bf06-4bb1c246fe84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:53.144686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec83bc-c466-47b6-bf06-4bb1c246fe84 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:53.144977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec83bc-c466-47b6-bf06-4bb1c246fe84 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:53.145258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec83bc-c466-47b6-bf06-4bb1c246fe84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:53.146993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-daec83bc-c466-47b6-bf06-4bb1c246fe84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:53.160978 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec83bc-c466-47b6-bf06-4bb1c246fe84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:53.161856 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec83bc-c466-47b6-bf06-4bb1c246fe84 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:53.173392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1367/2763] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:46:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 09:46:56.963719 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05aa3c-bca5-4e58-841d-a6ef7463b385 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:56.975791 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05aa3c-bca5-4e58-841d-a6ef7463b385 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:56.981190 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:56.982146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:56.982497 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:56.982863 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] PATH_INFO: `/v3/users/b432b13ffa7d42379fe38d6eed54e75f` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:56.984111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b432b13ffa7d42379fe38d6eed54e75f)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:56.986081 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:57.222665 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b432b13ffa7d42379fe38d6eed54e75f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:57.334420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b432b13ffa7d42379fe38d6eed54e75f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:57.335102 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b432b13ffa7d42379fe38d6eed54e75f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:57.343506 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b432b13ffa7d42379fe38d6eed54e75f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:57.348963 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-5b612ff3-99a9-4aa2-84bc-a6257e464ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b432b13ffa7d42379fe38d6eed54e75f'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:57.362377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1397/2764] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:46:56 2022] DELETE /identity/v3/users/b432b13ffa7d42379fe38d6eed54e75f => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:46:57.674689 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec83bc-c466-47b6-bf06-4bb1c246fe84 service nova] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:57.696738 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec83bc-c466-47b6-bf06-4bb1c246fe84 service nova] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:57.780217 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:57.781046 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:57.781383 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:57.781667 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] PATH_INFO: `/v3/projects/428f2755baac4091a9d78e86090d819b` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:57.782929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=428f2755baac4091a9d78e86090d819b)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:57.785127 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:57.895561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428f2755baac4091a9d78e86090d819b', 'request_id': 'req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:57.902133 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428f2755baac4091a9d78e86090d819b', 'request_id': 'req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:57.910666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428f2755baac4091a9d78e86090d819b', 'request_id': 'req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:58.009129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] The token cache is being invalidate because project 428f2755baac4091a9d78e86090d819b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:46:58.009444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:58.010061 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-9534fb15-cfbc-454e-adf1-9bd8ca053ac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:46:58.011675 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1368/2765] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:46:57 2022] DELETE /identity/v3/projects/428f2755baac4091a9d78e86090d819b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:46:58.017951 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ea577-0b66-46ca-8627-7250b8fb4b7a None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.018424 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ea577-0b66-46ca-8627-7250b8fb4b7a None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.018861 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ea577-0b66-46ca-8627-7250b8fb4b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.040400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-fe4ea577-0b66-46ca-8627-7250b8fb4b7a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:46:58.040400 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fe4ea577-0b66-46ca-8627-7250b8fb4b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:58.119254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1398/2766] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:46:58.128150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a25ff9-9823-48c4-9e8b-ddc13502bb5b None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.128453 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a25ff9-9823-48c4-9e8b-ddc13502bb5b None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.128736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a25ff9-9823-48c4-9e8b-ddc13502bb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.150784 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-27a25ff9-9823-48c4-9e8b-ddc13502bb5b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:46:58.152082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-27a25ff9-9823-48c4-9e8b-ddc13502bb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:58.160593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1369/2767] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:46:58.164801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ea577-0b66-46ca-8627-7250b8fb4b7a None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:58.176245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ea577-0b66-46ca-8627-7250b8fb4b7a None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:58.183468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5e8a7-83a9-425e-b1c6-48da7c244ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:58.183468 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5e8a7-83a9-425e-b1c6-48da7c244ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.184423 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5e8a7-83a9-425e-b1c6-48da7c244ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.184802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5e8a7-83a9-425e-b1c6-48da7c244ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.185692 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5e8a7-83a9-425e-b1c6-48da7c244ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:46:58.186282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5e8a7-83a9-425e-b1c6-48da7c244ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:58.187999 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5e8a7-83a9-425e-b1c6-48da7c244ea7 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:58.200564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1399/2768] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 09:46:58.205047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a25ff9-9823-48c4-9e8b-ddc13502bb5b None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:58.214997 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a25ff9-9823-48c4-9e8b-ddc13502bb5b None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:58.220557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6542e15-04ac-4f82-aa39-e5acf646d38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:58.221645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6542e15-04ac-4f82-aa39-e5acf646d38b admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.221645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6542e15-04ac-4f82-aa39-e5acf646d38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.221838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6542e15-04ac-4f82-aa39-e5acf646d38b admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.222530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6542e15-04ac-4f82-aa39-e5acf646d38b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:58.224958 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6542e15-04ac-4f82-aa39-e5acf646d38b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:58.322150 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1370/2769] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:46:58 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:46:58.326879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5e8a7-83a9-425e-b1c6-48da7c244ea7 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:58.338347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5e8a7-83a9-425e-b1c6-48da7c244ea7 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:58.391638 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:46:58.404672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad64a8a-3f03-4b80-82d8-1075b30e42ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:58.405433 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64a8a-3f03-4b80-82d8-1075b30e42ec admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.405789 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64a8a-3f03-4b80-82d8-1075b30e42ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.406259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64a8a-3f03-4b80-82d8-1075b30e42ec admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.407134 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad64a8a-3f03-4b80-82d8-1075b30e42ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:58.409198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad64a8a-3f03-4b80-82d8-1075b30e42ec admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:58.504366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1400/2770] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:46:58 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:46:58.508797 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6542e15-04ac-4f82-aa39-e5acf646d38b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:58.519800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6542e15-04ac-4f82-aa39-e5acf646d38b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:58.525009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c6c60-49c6-4412-8a90-062d4f110dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:58.525695 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c6c60-49c6-4412-8a90-062d4f110dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.525979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c6c60-49c6-4412-8a90-062d4f110dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.526255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c6c60-49c6-4412-8a90-062d4f110dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.527020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c6c60-49c6-4412-8a90-062d4f110dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:58.529221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c6c60-49c6-4412-8a90-062d4f110dbe admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:58.543793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1371/2771] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:46:58.545526 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad64a8a-3f03-4b80-82d8-1075b30e42ec admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:58.556122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad64a8a-3f03-4b80-82d8-1075b30e42ec admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:58.560924 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b42fe1-8cf3-49a2-adc7-e9849e011e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:58.561591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b42fe1-8cf3-49a2-adc7-e9849e011e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.561938 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b42fe1-8cf3-49a2-adc7-e9849e011e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.562503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b42fe1-8cf3-49a2-adc7-e9849e011e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.563472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b42fe1-8cf3-49a2-adc7-e9849e011e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:58.565614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b42fe1-8cf3-49a2-adc7-e9849e011e03 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:58.578554 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1401/2772] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:46:58.580848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c6c60-49c6-4412-8a90-062d4f110dbe admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:58.591754 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c6c60-49c6-4412-8a90-062d4f110dbe admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:58.596472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892535a-04d9-4957-b367-26cec030eefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:58.597368 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892535a-04d9-4957-b367-26cec030eefa admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.597653 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892535a-04d9-4957-b367-26cec030eefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.597959 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892535a-04d9-4957-b367-26cec030eefa admin admin] PATH_INFO: `/v3/projects/cf6354961eda4916a8324508b45a98d5/users/ca82a093a28f40f8a2bfc1f93df03493/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.614111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892535a-04d9-4957-b367-26cec030eefa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6354961eda4916a8324508b45a98d5, user_id=ca82a093a28f40f8a2bfc1f93df03493, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:58.617033 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892535a-04d9-4957-b367-26cec030eefa admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:58.718806 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1372/2773] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:46:58 2022] PUT /identity/v3/projects/cf6354961eda4916a8324508b45a98d5/users/ca82a093a28f40f8a2bfc1f93df03493/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:46:58.723935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03955e-d34d-4298-8dec-10f46387c76b None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.724230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03955e-d34d-4298-8dec-10f46387c76b None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.724511 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03955e-d34d-4298-8dec-10f46387c76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.742768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4f03955e-d34d-4298-8dec-10f46387c76b None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:46:58.743996 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4f03955e-d34d-4298-8dec-10f46387c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:58.800591 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1402/2774] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:46:58.810043 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892535a-04d9-4957-b367-26cec030eefa admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:46:58.811756 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-d892535a-04d9-4957-b367-26cec030eefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:46:58.825781 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:46:58.867988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892535a-04d9-4957-b367-26cec030eefa admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:46:58.871744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1dbb0-d641-48f2-8511-adaca1a9c578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:46:58.872377 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1dbb0-d641-48f2-8511-adaca1a9c578 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:46:58.872655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1dbb0-d641-48f2-8511-adaca1a9c578 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:46:58.872927 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1dbb0-d641-48f2-8511-adaca1a9c578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:46:58.881711 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1dbb0-d641-48f2-8511-adaca1a9c578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:46:58.883113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1dbb0-d641-48f2-8511-adaca1a9c578 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:46:58.892943 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1373/2775] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:47:05.432019 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3138e-70ff-4125-b7bc-7949690b3bf9 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:05.432522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3138e-70ff-4125-b7bc-7949690b3bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:05.432637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3138e-70ff-4125-b7bc-7949690b3bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:05.469988 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-e5d3138e-70ff-4125-b7bc-7949690b3bf9 None None] MFA Rules not processed for user `ca82a093a28f40f8a2bfc1f93df03493`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:47:05.471336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e5d3138e-70ff-4125-b7bc-7949690b3bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:05.561090 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1403/2776] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:47:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:47:05.562620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1dbb0-d641-48f2-8511-adaca1a9c578 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:05.573552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1dbb0-d641-48f2-8511-adaca1a9c578 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:05.578950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567385c2-f4b7-4660-981b-950d5036df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:05.580070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567385c2-f4b7-4660-981b-950d5036df30 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:05.580358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567385c2-f4b7-4660-981b-950d5036df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:05.580634 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567385c2-f4b7-4660-981b-950d5036df30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:05.581373 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567385c2-f4b7-4660-981b-950d5036df30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:05.583304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567385c2-f4b7-4660-981b-950d5036df30 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:05.673642 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1374/2777] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:47:05 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:47:05.674867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3138e-70ff-4125-b7bc-7949690b3bf9 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:05.688482 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3138e-70ff-4125-b7bc-7949690b3bf9 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:05.762184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:05.775122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109296d7-9675-4fe0-8ac3-fbd4ed843aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:05.778080 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109296d7-9675-4fe0-8ac3-fbd4ed843aec admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:05.778372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109296d7-9675-4fe0-8ac3-fbd4ed843aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:05.778655 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109296d7-9675-4fe0-8ac3-fbd4ed843aec admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:05.779440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109296d7-9675-4fe0-8ac3-fbd4ed843aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:05.781699 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109296d7-9675-4fe0-8ac3-fbd4ed843aec admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:05.926668 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1404/2778] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:47:05 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:47:05.932193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567385c2-f4b7-4660-981b-950d5036df30 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:05.942289 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567385c2-f4b7-4660-981b-950d5036df30 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:05.947097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b778b6f-07d7-42ec-83c9-1a4b821e39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:05.948002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b778b6f-07d7-42ec-83c9-1a4b821e39fc admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:05.948505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b778b6f-07d7-42ec-83c9-1a4b821e39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:05.948848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b778b6f-07d7-42ec-83c9-1a4b821e39fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:05.949709 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b778b6f-07d7-42ec-83c9-1a4b821e39fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:05.951852 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b778b6f-07d7-42ec-83c9-1a4b821e39fc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:05.967453 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1375/2779] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:47:05.971797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109296d7-9675-4fe0-8ac3-fbd4ed843aec admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:05.983886 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109296d7-9675-4fe0-8ac3-fbd4ed843aec admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:05.989684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd388fc7-b412-44d5-ab1d-c0b6a065dda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:05.990666 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd388fc7-b412-44d5-ab1d-c0b6a065dda4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:05.992266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd388fc7-b412-44d5-ab1d-c0b6a065dda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:05.992266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd388fc7-b412-44d5-ab1d-c0b6a065dda4 admin admin] PATH_INFO: `/v3/projects/441a9b972f8b4bcdbdabf9162f1e979b/users/9b9ea01d4a4f456d88c6dbeeb7b541df/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:06.008420 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd388fc7-b412-44d5-ab1d-c0b6a065dda4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441a9b972f8b4bcdbdabf9162f1e979b, user_id=9b9ea01d4a4f456d88c6dbeeb7b541df, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:06.011405 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd388fc7-b412-44d5-ab1d-c0b6a065dda4 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:06.098864 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1405/2780] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:47:05 2022] PUT /identity/v3/projects/441a9b972f8b4bcdbdabf9162f1e979b/users/9b9ea01d4a4f456d88c6dbeeb7b541df/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:47:06.109895 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b778b6f-07d7-42ec-83c9-1a4b821e39fc admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:06.120084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b778b6f-07d7-42ec-83c9-1a4b821e39fc admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:06.179891 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32b16-a8b3-4b2e-ae23-e43c70cdcb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:06.180560 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32b16-a8b3-4b2e-ae23-e43c70cdcb5c admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:06.180847 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32b16-a8b3-4b2e-ae23-e43c70cdcb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:06.181194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32b16-a8b3-4b2e-ae23-e43c70cdcb5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:06.182042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c32b16-a8b3-4b2e-ae23-e43c70cdcb5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:06.183844 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c32b16-a8b3-4b2e-ae23-e43c70cdcb5c admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:06.195455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1376/2781] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:47:06.199734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd388fc7-b412-44d5-ab1d-c0b6a065dda4 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:06.212936 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd388fc7-b412-44d5-ab1d-c0b6a065dda4 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:06.220115 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b6889-cfd2-455f-87a6-30170ca4cebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:06.225926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6889-cfd2-455f-87a6-30170ca4cebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:06.225926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6889-cfd2-455f-87a6-30170ca4cebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:06.225926 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6889-cfd2-455f-87a6-30170ca4cebe admin admin] PATH_INFO: `/v3/projects/441a9b972f8b4bcdbdabf9162f1e979b/users/9b9ea01d4a4f456d88c6dbeeb7b541df/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:06.228667 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b6889-cfd2-455f-87a6-30170ca4cebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=441a9b972f8b4bcdbdabf9162f1e979b, user_id=9b9ea01d4a4f456d88c6dbeeb7b541df, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:06.232088 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b6889-cfd2-455f-87a6-30170ca4cebe admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:06.246871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1406/2782] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:47:06 2022] PUT /identity/v3/projects/441a9b972f8b4bcdbdabf9162f1e979b/users/9b9ea01d4a4f456d88c6dbeeb7b541df/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:47:07.645492 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32b16-a8b3-4b2e-ae23-e43c70cdcb5c admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:07.647786 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f7c32b16-a8b3-4b2e-ae23-e43c70cdcb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:07.753422 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32b16-a8b3-4b2e-ae23-e43c70cdcb5c admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:07.842208 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:07.881143 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:07.882684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:07.882909 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:07.883191 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:07.886462 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:08.032974 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:08.035087 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:08.058619 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1377/2783] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:47:08.305333 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46e538-8028-4af5-a636-01ace86246dc None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:08.305973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46e538-8028-4af5-a636-01ace86246dc None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:08.305973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46e538-8028-4af5-a636-01ace86246dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:08.350404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-4a46e538-8028-4af5-a636-01ace86246dc None None] MFA Rules not processed for user `222b727863844d2b95a9c7222a56a94c`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:47:08.356955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4a46e538-8028-4af5-a636-01ace86246dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:08.449913 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1407/2784] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:47:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:47:13.657177 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:13.661071 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:13.795685 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277424c9-e96c-4f32-adb3-5503f195a4e0 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:13.809379 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:13.810162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:13.810818 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:13.812192 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:13.814829 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:13.834255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:13.834255 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:13.873906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1378/2785] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:47:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:47:15.594074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077485ee-ef37-47ca-a95f-f7f61ccfebe6 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:15.594614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077485ee-ef37-47ca-a95f-f7f61ccfebe6 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:15.594762 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077485ee-ef37-47ca-a95f-f7f61ccfebe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:15.620869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-077485ee-ef37-47ca-a95f-f7f61ccfebe6 None None] MFA Rules not processed for user `9b9ea01d4a4f456d88c6dbeeb7b541df`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:47:15.622089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-077485ee-ef37-47ca-a95f-f7f61ccfebe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:15.704800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1408/2786] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:47:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:47:15.719145 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:15.720595 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:15.821222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf044d-27d7-4eee-b429-44eef19f3033 service placement] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:15.839573 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445bd8f-0434-4fef-8aff-6d153f2a7465 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:15.840429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445bd8f-0434-4fef-8aff-6d153f2a7465 service nova] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:15.841126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445bd8f-0434-4fef-8aff-6d153f2a7465 service nova] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:15.841549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445bd8f-0434-4fef-8aff-6d153f2a7465 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:15.862990 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:15.953148 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445bd8f-0434-4fef-8aff-6d153f2a7465 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:15.955079 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445bd8f-0434-4fef-8aff-6d153f2a7465 service nova] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:15.990830 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1379/2787] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:47:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:47:19.150583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec334a69-e7cf-4b75-bdb5-bd37c556f7e4 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:19.154460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec334a69-e7cf-4b75-bdb5-bd37c556f7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:19.154460 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec334a69-e7cf-4b75-bdb5-bd37c556f7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:19.169289 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-ec334a69-e7cf-4b75-bdb5-bd37c556f7e4 None None] MFA Rules not processed for user `ca82a093a28f40f8a2bfc1f93df03493`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:47:19.170502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ec334a69-e7cf-4b75-bdb5-bd37c556f7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:19.177693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1409/2788] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:47:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:47:19.492967 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445bd8f-0434-4fef-8aff-6d153f2a7465 service nova] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:19.508779 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445bd8f-0434-4fef-8aff-6d153f2a7465 service nova] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:19.516423 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451f5e9-7e6f-4db9-adfb-fc35cc65871f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:19.518557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451f5e9-7e6f-4db9-adfb-fc35cc65871f service placement] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:19.519002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451f5e9-7e6f-4db9-adfb-fc35cc65871f service placement] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:19.519453 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451f5e9-7e6f-4db9-adfb-fc35cc65871f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:19.522684 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3451f5e9-7e6f-4db9-adfb-fc35cc65871f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:19.538279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451f5e9-7e6f-4db9-adfb-fc35cc65871f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:19.545433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451f5e9-7e6f-4db9-adfb-fc35cc65871f service placement] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:19.564440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1380/2789] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:47:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:47:19.688729 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec334a69-e7cf-4b75-bdb5-bd37c556f7e4 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:19.690148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-ec334a69-e7cf-4b75-bdb5-bd37c556f7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:19.706531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec334a69-e7cf-4b75-bdb5-bd37c556f7e4 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:19.711077 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:19.711997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:19.712278 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:19.712553 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] PATH_INFO: `/v3/users/2720d45854ab4a61b3b22dd43a523d4b` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:19.713867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2720d45854ab4a61b3b22dd43a523d4b)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:19.715903 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:19.858807 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:19.991833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2720d45854ab4a61b3b22dd43a523d4b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.063262 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2720d45854ab4a61b3b22dd43a523d4b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.064839 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2720d45854ab4a61b3b22dd43a523d4b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.074624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2720d45854ab4a61b3b22dd43a523d4b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.077919 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2720d45854ab4a61b3b22dd43a523d4b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.093297 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1410/2790] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:47:19 2022] DELETE /identity/v3/users/2720d45854ab4a61b3b22dd43a523d4b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:47:20.410698 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451f5e9-7e6f-4db9-adfb-fc35cc65871f service placement] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:20.423472 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451f5e9-7e6f-4db9-adfb-fc35cc65871f service placement] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:20.482275 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:20.483146 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:20.483430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:20.483760 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] PATH_INFO: `/v3/projects/1d5cbd080e204408aa51c29ec2cf1422` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:20.484931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d5cbd080e204408aa51c29ec2cf1422)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:20.486840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:20.633606 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d5cbd080e204408aa51c29ec2cf1422', 'request_id': 'req-987597e1-3f30-4162-8047-3d7b58d72494'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.640126 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d5cbd080e204408aa51c29ec2cf1422', 'request_id': 'req-987597e1-3f30-4162-8047-3d7b58d72494'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.652297 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d5cbd080e204408aa51c29ec2cf1422', 'request_id': 'req-987597e1-3f30-4162-8047-3d7b58d72494'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.747963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] The token cache is being invalidate because project 1d5cbd080e204408aa51c29ec2cf1422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:47:20.748258 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.748938 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:20.750575 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1381/2791] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:47:20 2022] DELETE /identity/v3/projects/1d5cbd080e204408aa51c29ec2cf1422 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:47:20.755022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:20.756637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:20.772357 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c0ceb-1c67-4430-9186-7ca66388309a admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:20.833550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:20.835002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:20.835299 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:20.835615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] PATH_INFO: `/v3/users/4987f547ce064257a0b180c83ef14bef` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:20.836893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4987f547ce064257a0b180c83ef14bef)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:20.838908 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:21.027618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4987f547ce064257a0b180c83ef14bef'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.088541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4987f547ce064257a0b180c83ef14bef'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.089135 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4987f547ce064257a0b180c83ef14bef'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.098141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4987f547ce064257a0b180c83ef14bef'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.098141 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4987f547ce064257a0b180c83ef14bef'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.116595 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1411/2792] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:47:20 2022] DELETE /identity/v3/users/4987f547ce064257a0b180c83ef14bef => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:47:21.379686 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:21.381153 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:21.397420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987597e1-3f30-4162-8047-3d7b58d72494 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:21.425767 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:21.457546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:21.458432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:21.458715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:21.458993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] PATH_INFO: `/v3/projects/cac420d7f4d849e7b14b7f0c37dbf3bc` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:21.460230 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cac420d7f4d849e7b14b7f0c37dbf3bc)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:21.462140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:21.586972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac420d7f4d849e7b14b7f0c37dbf3bc', 'request_id': 'req-15bd7718-8a84-4863-b40c-59076d8c45a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.592282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac420d7f4d849e7b14b7f0c37dbf3bc', 'request_id': 'req-15bd7718-8a84-4863-b40c-59076d8c45a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.599265 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac420d7f4d849e7b14b7f0c37dbf3bc', 'request_id': 'req-15bd7718-8a84-4863-b40c-59076d8c45a4'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.679467 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] The token cache is being invalidate because project cac420d7f4d849e7b14b7f0c37dbf3bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:47:21.680159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.680290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:47:21.681916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1382/2793] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:47:21 2022] DELETE /identity/v3/projects/cac420d7f4d849e7b14b7f0c37dbf3bc => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:47:22.518236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:22.519788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:22.551311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:22.594150 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4086ff1-7a1a-4f87-aecc-84d090380a8c admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:22.663451 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:22.664132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:22.664415 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:22.664689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:22.666470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:22.679187 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:22.680900 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:22.695198 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1412/2794] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:47:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:47:31.676226 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:32.734307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=52 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:35.204864 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:35.206315 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:35.261947 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd7718-8a84-4863-b40c-59076d8c45a4 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:35.265574 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:35.266213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:35.266497 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:35.266776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:35.268568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:35.278654 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:35.280122 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:35.290530 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1383/2795] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:47:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:47:35.994952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:35.996436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:36.066386 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad6aee-7203-404c-ad6c-7dd28c3a9434 service nova] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:36.071359 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:36.072098 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:36.072323 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:36.072601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:36.074392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:36.086336 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:36.087954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:36.098183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1413/2796] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:47:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:47:45.284452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=26 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:46.093997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=32 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:47:53.727437 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:53.728880 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:53.739091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698df69f-f6ce-45eb-8b77-f81e646fe731 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:53.742723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:53.743358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:53.743676 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:53.743955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:53.745666 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:53.807156 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:53.808553 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:53.818496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1384/2797] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:47:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:47:55.884452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:47:55.886426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:55.902592 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5ce2e-bf83-4354-9d1b-e43ee8d8840f service placement] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:47:55.908378 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5041d2-beef-4503-bff6-d0a4ff53c8d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:47:55.908973 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5041d2-beef-4503-bff6-d0a4ff53c8d4 service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:47:55.909255 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5041d2-beef-4503-bff6-d0a4ff53c8d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:47:55.909535 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5041d2-beef-4503-bff6-d0a4ff53c8d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:47:55.911310 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-4a5041d2-beef-4503-bff6-d0a4ff53c8d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:47:55.924099 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5041d2-beef-4503-bff6-d0a4ff53c8d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:47:55.925772 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5041d2-beef-4503-bff6-d0a4ff53c8d4 service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:47:55.937383 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1414/2798] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:47:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:48:03.814311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=30 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:05.932243 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=6 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:40.191602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:40.194860 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:40.324955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e89fa7-44e4-4ee8-b2c0-03fa33727acf service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:40.329420 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:40.330283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:40.330559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:40.330832 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] PATH_INFO: `/v3/users/222b727863844d2b95a9c7222a56a94c` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:40.332082 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=222b727863844d2b95a9c7222a56a94c)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:40.334010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:40.492229 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '222b727863844d2b95a9c7222a56a94c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:40.492888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '222b727863844d2b95a9c7222a56a94c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:40.499773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '222b727863844d2b95a9c7222a56a94c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:40.500430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '222b727863844d2b95a9c7222a56a94c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:40.513077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6aabcbe3-c6c0-46ff-bc2f-9c2c3aee27a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '222b727863844d2b95a9c7222a56a94c'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:40.575083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1385/2799] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:48:40 2022] DELETE /identity/v3/users/222b727863844d2b95a9c7222a56a94c => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:40.845307 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5041d2-beef-4503-bff6-d0a4ff53c8d4 service nova] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:40.882876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5041d2-beef-4503-bff6-d0a4ff53c8d4 service nova] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:40.937121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:40.937975 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:40.938257 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:40.938533 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] PATH_INFO: `/v3/projects/422340b9e5834d16bff3539c549fc4c1` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:40.939730 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=422340b9e5834d16bff3539c549fc4c1)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:40.941691 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:41.036012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '422340b9e5834d16bff3539c549fc4c1', 'request_id': 'req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:41.092639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '422340b9e5834d16bff3539c549fc4c1', 'request_id': 'req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:41.098752 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '422340b9e5834d16bff3539c549fc4c1', 'request_id': 'req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:41.109541 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] The token cache is being invalidate because project 422340b9e5834d16bff3539c549fc4c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:48:41.109823 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:41.110447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-dbe8f155-5c9e-46b3-8d37-a73c3b0cd7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:41.111997 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1415/2800] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:48:40 2022] DELETE /identity/v3/projects/422340b9e5834d16bff3539c549fc4c1 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:41.116962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f37994-0e32-40eb-b589-1a90495222f4 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:41.117256 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f37994-0e32-40eb-b589-1a90495222f4 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:41.117532 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f37994-0e32-40eb-b589-1a90495222f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:41.139563 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:41.175875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-27f37994-0e32-40eb-b589-1a90495222f4 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:41.177152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-27f37994-0e32-40eb-b589-1a90495222f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:41.235683 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1386/2801] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:41.242978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caf52e-08a4-4753-8a97-b2c970b95304 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:41.243272 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caf52e-08a4-4753-8a97-b2c970b95304 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:41.243584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caf52e-08a4-4753-8a97-b2c970b95304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:41.261495 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-00caf52e-08a4-4753-8a97-b2c970b95304 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:41.262694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-00caf52e-08a4-4753-8a97-b2c970b95304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:41.271153 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1416/2802] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:41.275278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f37994-0e32-40eb-b589-1a90495222f4 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:41.285599 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f37994-0e32-40eb-b589-1a90495222f4 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:41.290214 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030d20-e9ba-489b-a9ca-ca0ec96891ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:41.290870 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030d20-e9ba-489b-a9ca-ca0ec96891ee admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:41.291149 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030d20-e9ba-489b-a9ca-ca0ec96891ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:41.291429 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030d20-e9ba-489b-a9ca-ca0ec96891ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:41.292196 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c030d20-e9ba-489b-a9ca-ca0ec96891ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:48:41.292501 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c030d20-e9ba-489b-a9ca-ca0ec96891ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:41.294129 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c030d20-e9ba-489b-a9ca-ca0ec96891ee admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:41.305867 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1387/2803] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:48:41.310416 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caf52e-08a4-4753-8a97-b2c970b95304 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:41.319800 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caf52e-08a4-4753-8a97-b2c970b95304 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:41.324477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96d1ee-4dec-4d10-8db1-a3cb88d26001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:41.325123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96d1ee-4dec-4d10-8db1-a3cb88d26001 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:41.325408 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96d1ee-4dec-4d10-8db1-a3cb88d26001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:41.325684 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96d1ee-4dec-4d10-8db1-a3cb88d26001 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:41.326369 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96d1ee-4dec-4d10-8db1-a3cb88d26001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:41.328308 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96d1ee-4dec-4d10-8db1-a3cb88d26001 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:41.440108 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1417/2804] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:48:41 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:48:41.444957 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030d20-e9ba-489b-a9ca-ca0ec96891ee admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:41.455243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030d20-e9ba-489b-a9ca-ca0ec96891ee admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:41.512777 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7dbd11-2366-4575-8725-586ee627360a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:41.513454 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dbd11-2366-4575-8725-586ee627360a admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:41.513728 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dbd11-2366-4575-8725-586ee627360a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:41.514006 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dbd11-2366-4575-8725-586ee627360a admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:41.514704 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7dbd11-2366-4575-8725-586ee627360a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:41.516656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7dbd11-2366-4575-8725-586ee627360a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:41.613178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1388/2805] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:48:41 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:48:41.617119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96d1ee-4dec-4d10-8db1-a3cb88d26001 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:41.627320 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96d1ee-4dec-4d10-8db1-a3cb88d26001 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:41.632550 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e67fd-a859-4d67-a0b1-241dc4de763e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:41.633146 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e67fd-a859-4d67-a0b1-241dc4de763e admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:41.633427 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e67fd-a859-4d67-a0b1-241dc4de763e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:41.633700 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e67fd-a859-4d67-a0b1-241dc4de763e admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:41.634447 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e67fd-a859-4d67-a0b1-241dc4de763e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:41.636078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e67fd-a859-4d67-a0b1-241dc4de763e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:41.646821 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1418/2806] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:41.650888 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7dbd11-2366-4575-8725-586ee627360a admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:41.661247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7dbd11-2366-4575-8725-586ee627360a admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:41.666023 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26073085-0183-4774-ac1c-11eea9b5dec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:41.671717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26073085-0183-4774-ac1c-11eea9b5dec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:41.671717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26073085-0183-4774-ac1c-11eea9b5dec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:41.671717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26073085-0183-4774-ac1c-11eea9b5dec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:41.671717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26073085-0183-4774-ac1c-11eea9b5dec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:41.671717 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26073085-0183-4774-ac1c-11eea9b5dec3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:41.680222 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1389/2807] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:41.684156 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e67fd-a859-4d67-a0b1-241dc4de763e admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:41.694585 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e67fd-a859-4d67-a0b1-241dc4de763e admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:41.700569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0da00-6ba4-4cd7-8bc1-893cfd4fe262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:41.700569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0da00-6ba4-4cd7-8bc1-893cfd4fe262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:41.700569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0da00-6ba4-4cd7-8bc1-893cfd4fe262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:41.700887 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0da00-6ba4-4cd7-8bc1-893cfd4fe262 admin admin] PATH_INFO: `/v3/projects/e0499021ec5e479db8de9c925c515910/users/a474031094074aaf8333df2414c37a0a/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:41.715898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0da00-6ba4-4cd7-8bc1-893cfd4fe262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0499021ec5e479db8de9c925c515910, user_id=a474031094074aaf8333df2414c37a0a, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:41.718531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0da00-6ba4-4cd7-8bc1-893cfd4fe262 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:41.812651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1419/2808] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:48:41 2022] PUT /identity/v3/projects/e0499021ec5e479db8de9c925c515910/users/a474031094074aaf8333df2414c37a0a/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:41.823053 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01858a36-ec91-4e2e-92f5-e29948c8a1fe None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:41.823344 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01858a36-ec91-4e2e-92f5-e29948c8a1fe None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:41.823655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01858a36-ec91-4e2e-92f5-e29948c8a1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:41.849292 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-01858a36-ec91-4e2e-92f5-e29948c8a1fe None None] MFA Rules not processed for user `a474031094074aaf8333df2414c37a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:41.850491 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-01858a36-ec91-4e2e-92f5-e29948c8a1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:41.893277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:41.915328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1390/2809] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:41.927303 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0da00-6ba4-4cd7-8bc1-893cfd4fe262 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:41.929030 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a0f0da00-6ba4-4cd7-8bc1-893cfd4fe262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:41.931994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:42.017974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0da00-6ba4-4cd7-8bc1-893cfd4fe262 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:42.023586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55005075-9849-4f08-b2bf-e0823fb1943f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e0fbe0b5814ad1ae95c65a64b632d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:42.024233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55005075-9849-4f08-b2bf-e0823fb1943f service nova] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:42.024514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55005075-9849-4f08-b2bf-e0823fb1943f service nova] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:42.024792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55005075-9849-4f08-b2bf-e0823fb1943f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:42.032505 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55005075-9849-4f08-b2bf-e0823fb1943f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:42.034164 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55005075-9849-4f08-b2bf-e0823fb1943f service nova] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:42.042879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1420/2810] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:48:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:48:42.252330 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01858a36-ec91-4e2e-92f5-e29948c8a1fe None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:42.316378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01858a36-ec91-4e2e-92f5-e29948c8a1fe None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:42.320907 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:42.321773 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:42.322053 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:42.322328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] PATH_INFO: `/v3/users/a474031094074aaf8333df2414c37a0a` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:42.323514 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a474031094074aaf8333df2414c37a0a)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:42.325453 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:42.550723 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a474031094074aaf8333df2414c37a0a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:42.551496 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a474031094074aaf8333df2414c37a0a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:42.558702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a474031094074aaf8333df2414c37a0a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:42.559360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a474031094074aaf8333df2414c37a0a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:42.571382 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a474031094074aaf8333df2414c37a0a'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:42.650243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1391/2811] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:48:42 2022] DELETE /identity/v3/users/a474031094074aaf8333df2414c37a0a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:42.654728 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ddff8-12a3-4505-907e-06b5c0124fdc None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:42.655037 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ddff8-12a3-4505-907e-06b5c0124fdc None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:42.655354 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ddff8-12a3-4505-907e-06b5c0124fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:42.673647 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-b00ddff8-12a3-4505-907e-06b5c0124fdc None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:42.674869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b00ddff8-12a3-4505-907e-06b5c0124fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:42.732151 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1421/2812] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:48:42.741602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:42.743003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:42.810774 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:42.819681 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b208789-a27c-4cd2-90fe-5bda8c73be99 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:42.823333 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78616fda-a440-4fb3-b7bc-37f698091bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:42.824014 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78616fda-a440-4fb3-b7bc-37f698091bbd service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:42.824300 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78616fda-a440-4fb3-b7bc-37f698091bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:42.824577 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78616fda-a440-4fb3-b7bc-37f698091bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:42.834238 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78616fda-a440-4fb3-b7bc-37f698091bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:42.835811 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78616fda-a440-4fb3-b7bc-37f698091bbd service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:42.845736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1392/2813] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:48:43.254751 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ddff8-12a3-4505-907e-06b5c0124fdc None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:43.256236 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b00ddff8-12a3-4505-907e-06b5c0124fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:43.275326 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ddff8-12a3-4505-907e-06b5c0124fdc None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:43.281530 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:43.282419 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:43.282754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:43.283065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] PATH_INFO: `/v3/projects/e0499021ec5e479db8de9c925c515910` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:43.284372 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0499021ec5e479db8de9c925c515910)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:43.286375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:43.436311 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0499021ec5e479db8de9c925c515910', 'request_id': 'req-2b1cc6a2-a9dd-4d85-b150-67218a51a270'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:43.504725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0499021ec5e479db8de9c925c515910', 'request_id': 'req-2b1cc6a2-a9dd-4d85-b150-67218a51a270'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:43.510418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0499021ec5e479db8de9c925c515910', 'request_id': 'req-2b1cc6a2-a9dd-4d85-b150-67218a51a270'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:43.512287 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:43.521453 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] The token cache is being invalidate because project e0499021ec5e479db8de9c925c515910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80345) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:48:43.521733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:43.522358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-2b1cc6a2-a9dd-4d85-b150-67218a51a270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:43.523917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1422/2814] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:48:43 2022] DELETE /identity/v3/projects/e0499021ec5e479db8de9c925c515910 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:43.528883 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c42aa-dd1b-42f4-88f1-4f748bd1f901 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:43.529358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c42aa-dd1b-42f4-88f1-4f748bd1f901 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:43.529470 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c42aa-dd1b-42f4-88f1-4f748bd1f901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:43.547590 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-eb8c42aa-dd1b-42f4-88f1-4f748bd1f901 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:43.548842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-eb8c42aa-dd1b-42f4-88f1-4f748bd1f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:43.607800 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1393/2815] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:43.617193 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653986ed-2ac4-4d6a-8edd-8a30091f1444 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:43.617580 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653986ed-2ac4-4d6a-8edd-8a30091f1444 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:43.617855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653986ed-2ac4-4d6a-8edd-8a30091f1444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:43.636736 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-653986ed-2ac4-4d6a-8edd-8a30091f1444 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:43.637940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-653986ed-2ac4-4d6a-8edd-8a30091f1444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:43.645848 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1423/2816] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:48:43.649966 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c42aa-dd1b-42f4-88f1-4f748bd1f901 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:43.660194 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c42aa-dd1b-42f4-88f1-4f748bd1f901 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:43.664737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940f2b3-f745-40fe-9445-95899b599394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:43.665386 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940f2b3-f745-40fe-9445-95899b599394 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:43.665671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940f2b3-f745-40fe-9445-95899b599394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:43.665968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940f2b3-f745-40fe-9445-95899b599394 admin admin] PATH_INFO: `/v3/domains` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:43.666700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940f2b3-f745-40fe-9445-95899b599394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80344) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:48:43.667010 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940f2b3-f745-40fe-9445-95899b599394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:43.668645 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940f2b3-f745-40fe-9445-95899b599394 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:43.680919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1394/2817] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:48:43.684899 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653986ed-2ac4-4d6a-8edd-8a30091f1444 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:43.694060 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653986ed-2ac4-4d6a-8edd-8a30091f1444 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:43.698597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67e087-19a1-4743-a01a-b581741fce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:43.699237 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67e087-19a1-4743-a01a-b581741fce66 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:43.699547 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67e087-19a1-4743-a01a-b581741fce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:43.699836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67e087-19a1-4743-a01a-b581741fce66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:43.700523 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67e087-19a1-4743-a01a-b581741fce66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:43.702404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67e087-19a1-4743-a01a-b581741fce66 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:43.780509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1424/2818] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:48:43 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:48:43.785137 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940f2b3-f745-40fe-9445-95899b599394 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:43.795236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940f2b3-f745-40fe-9445-95899b599394 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:43.851625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02df6-2059-4941-93e2-a0f3a0bede5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:43.852278 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02df6-2059-4941-93e2-a0f3a0bede5a admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:43.852559 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02df6-2059-4941-93e2-a0f3a0bede5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:43.852833 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef02df6-2059-4941-93e2-a0f3a0bede5a admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:43.853510 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef02df6-2059-4941-93e2-a0f3a0bede5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:43.855409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef02df6-2059-4941-93e2-a0f3a0bede5a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:43.930881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1395/2819] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:48:43 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:48:43.934751 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67e087-19a1-4743-a01a-b581741fce66 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:43.945285 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67e087-19a1-4743-a01a-b581741fce66 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:43.949833 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfd123-9921-4d80-adb8-b8db1e2f0fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:43.950481 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfd123-9921-4d80-adb8-b8db1e2f0fae admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:43.950764 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfd123-9921-4d80-adb8-b8db1e2f0fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:43.951065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfd123-9921-4d80-adb8-b8db1e2f0fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:43.951792 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfd123-9921-4d80-adb8-b8db1e2f0fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:43.953379 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfd123-9921-4d80-adb8-b8db1e2f0fae admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:43.963514 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1425/2820] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:43.967331 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02df6-2059-4941-93e2-a0f3a0bede5a admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:43.977941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef02df6-2059-4941-93e2-a0f3a0bede5a admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:43.982600 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9f3bd-02fd-4b37-9831-aff47e5ba105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:43.983249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9f3bd-02fd-4b37-9831-aff47e5ba105 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:43.983552 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9f3bd-02fd-4b37-9831-aff47e5ba105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:43.983846 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9f3bd-02fd-4b37-9831-aff47e5ba105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:43.984571 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9f3bd-02fd-4b37-9831-aff47e5ba105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:43.986164 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9f3bd-02fd-4b37-9831-aff47e5ba105 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:43.998209 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1396/2821] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:44.002233 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfd123-9921-4d80-adb8-b8db1e2f0fae admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:44.012286 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfd123-9921-4d80-adb8-b8db1e2f0fae admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:44.017104 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98988581-2819-4845-a521-39bdb116cd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:44.017970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98988581-2819-4845-a521-39bdb116cd22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:44.018282 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98988581-2819-4845-a521-39bdb116cd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:44.018562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98988581-2819-4845-a521-39bdb116cd22 admin admin] PATH_INFO: `/v3/projects/302572b968444988af8811651f52d6ea/users/6de826e19dcc4de7bec6ba650e476c8b/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:44.032725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98988581-2819-4845-a521-39bdb116cd22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=302572b968444988af8811651f52d6ea, user_id=6de826e19dcc4de7bec6ba650e476c8b, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:44.035940 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98988581-2819-4845-a521-39bdb116cd22 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:44.100025 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1426/2822] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:48:44 2022] PUT /identity/v3/projects/302572b968444988af8811651f52d6ea/users/6de826e19dcc4de7bec6ba650e476c8b/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:44.116095 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26644e6-5c15-4ae6-b580-c4cddf9836a6 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:44.116388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26644e6-5c15-4ae6-b580-c4cddf9836a6 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:44.116669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26644e6-5c15-4ae6-b580-c4cddf9836a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:44.142227 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-a26644e6-5c15-4ae6-b580-c4cddf9836a6 None None] MFA Rules not processed for user `6de826e19dcc4de7bec6ba650e476c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:44.143954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-a26644e6-5c15-4ae6-b580-c4cddf9836a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:44.182948 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:44.206697 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1397/2823] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:48:44.211004 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98988581-2819-4845-a521-39bdb116cd22 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:44.273502 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98988581-2819-4845-a521-39bdb116cd22 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:44.278962 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e0143-202c-46c5-ae79-b07367615df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:44.279656 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e0143-202c-46c5-ae79-b07367615df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:44.279944 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e0143-202c-46c5-ae79-b07367615df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:44.280222 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e0143-202c-46c5-ae79-b07367615df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:44.280925 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e0143-202c-46c5-ae79-b07367615df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:44.282945 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e0143-202c-46c5-ae79-b07367615df5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:44.364003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1427/2824] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:48:44 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:48:44.368063 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26644e6-5c15-4ae6-b580-c4cddf9836a6 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:44.378712 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26644e6-5c15-4ae6-b580-c4cddf9836a6 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:44.434022 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554516b-db17-490b-893d-349dd64f4ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:44.434690 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554516b-db17-490b-893d-349dd64f4ade admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:44.434968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554516b-db17-490b-893d-349dd64f4ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:44.435243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554516b-db17-490b-893d-349dd64f4ade admin admin] PATH_INFO: `/v3/users` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:44.435968 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554516b-db17-490b-893d-349dd64f4ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:44.437894 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554516b-db17-490b-893d-349dd64f4ade admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:44.552929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1398/2825] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:48:44 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:48:44.556948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e0143-202c-46c5-ae79-b07367615df5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:44.568056 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e0143-202c-46c5-ae79-b07367615df5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:44.572650 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afcedf-69c0-4f19-bca9-c9e09d18b7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:44.573294 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afcedf-69c0-4f19-bca9-c9e09d18b7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:44.573574 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afcedf-69c0-4f19-bca9-c9e09d18b7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:44.573896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afcedf-69c0-4f19-bca9-c9e09d18b7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:44.574612 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afcedf-69c0-4f19-bca9-c9e09d18b7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:44.576268 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afcedf-69c0-4f19-bca9-c9e09d18b7f8 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:44.588079 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1428/2826] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:44.592000 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554516b-db17-490b-893d-349dd64f4ade admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:44.604274 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554516b-db17-490b-893d-349dd64f4ade admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:44.608961 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340fff9-36ff-4023-b039-a323ade8d75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:44.609835 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340fff9-36ff-4023-b039-a323ade8d75b admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:44.610116 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340fff9-36ff-4023-b039-a323ade8d75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:44.610393 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340fff9-36ff-4023-b039-a323ade8d75b admin admin] PATH_INFO: `/v3/projects/694dc0f8bbdf44d495874455165ddc95/users/19b603f7577546c08ed754c2a0a354c7/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:44.624857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340fff9-36ff-4023-b039-a323ade8d75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=694dc0f8bbdf44d495874455165ddc95, user_id=19b603f7577546c08ed754c2a0a354c7, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:44.627516 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340fff9-36ff-4023-b039-a323ade8d75b admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:44.723887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1399/2827] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:48:44 2022] PUT /identity/v3/projects/694dc0f8bbdf44d495874455165ddc95/users/19b603f7577546c08ed754c2a0a354c7/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:48:44.727757 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afcedf-69c0-4f19-bca9-c9e09d18b7f8 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:44.737994 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afcedf-69c0-4f19-bca9-c9e09d18b7f8 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:44.792923 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ece89-5197-427f-9c91-01523a8796b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:44.793586 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ece89-5197-427f-9c91-01523a8796b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:44.793870 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ece89-5197-427f-9c91-01523a8796b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:44.794148 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ece89-5197-427f-9c91-01523a8796b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:44.794868 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ece89-5197-427f-9c91-01523a8796b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:44.796510 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ece89-5197-427f-9c91-01523a8796b1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:44.806775 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1429/2828] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:44.810884 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340fff9-36ff-4023-b039-a323ade8d75b admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:44.821809 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340fff9-36ff-4023-b039-a323ade8d75b admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:44.826440 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2651a-c70d-4208-a82b-bc79edd70ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:44.827321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2651a-c70d-4208-a82b-bc79edd70ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:44.827631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2651a-c70d-4208-a82b-bc79edd70ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:44.827918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2651a-c70d-4208-a82b-bc79edd70ef7 admin admin] PATH_INFO: `/v3/projects/694dc0f8bbdf44d495874455165ddc95/users/19b603f7577546c08ed754c2a0a354c7/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:44.830007 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2651a-c70d-4208-a82b-bc79edd70ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694dc0f8bbdf44d495874455165ddc95, user_id=19b603f7577546c08ed754c2a0a354c7, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:44.832701 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2651a-c70d-4208-a82b-bc79edd70ef7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:44.843941 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1400/2829] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:48:44 2022] PUT /identity/v3/projects/694dc0f8bbdf44d495874455165ddc95/users/19b603f7577546c08ed754c2a0a354c7/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:44.853777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddecd38-3203-4204-b4f2-227182ad117d None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:44.854067 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddecd38-3203-4204-b4f2-227182ad117d None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:44.854340 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddecd38-3203-4204-b4f2-227182ad117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:44.860124 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:44.881241 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-eddecd38-3203-4204-b4f2-227182ad117d None None] MFA Rules not processed for user `19b603f7577546c08ed754c2a0a354c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:44.882543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-eddecd38-3203-4204-b4f2-227182ad117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:44.949836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1430/2830] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:44.961055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2651a-c70d-4208-a82b-bc79edd70ef7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:44.962736 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-e5a2651a-c70d-4208-a82b-bc79edd70ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:45.019735 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2651a-c70d-4208-a82b-bc79edd70ef7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:45.023354 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1736fa-d032-4039-aa9a-7bc496a86563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:45.024039 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1736fa-d032-4039-aa9a-7bc496a86563 service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:45.024321 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1736fa-d032-4039-aa9a-7bc496a86563 service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:45.024597 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1736fa-d032-4039-aa9a-7bc496a86563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:45.033764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1736fa-d032-4039-aa9a-7bc496a86563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:45.035271 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1736fa-d032-4039-aa9a-7bc496a86563 service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:45.044783 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1401/2831] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:48:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:48:45.917966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddecd38-3203-4204-b4f2-227182ad117d None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:45.983438 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddecd38-3203-4204-b4f2-227182ad117d None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:45.988038 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:45.988895 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:45.989174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:45.989443 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] PATH_INFO: `/v3/users/6de826e19dcc4de7bec6ba650e476c8b` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:45.990615 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de826e19dcc4de7bec6ba650e476c8b)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:45.992571 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:46.223215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de826e19dcc4de7bec6ba650e476c8b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:46.295680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de826e19dcc4de7bec6ba650e476c8b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:46.296304 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de826e19dcc4de7bec6ba650e476c8b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:46.304776 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de826e19dcc4de7bec6ba650e476c8b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:46.305402 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de826e19dcc4de7bec6ba650e476c8b'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:46.323112 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1431/2832] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:48:45 2022] DELETE /identity/v3/users/6de826e19dcc4de7bec6ba650e476c8b => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:46.327625 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad838-d779-497a-b9a0-d87844e267af None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:46.328083 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad838-d779-497a-b9a0-d87844e267af None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:46.328235 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad838-d779-497a-b9a0-d87844e267af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:46.345531 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-7c8ad838-d779-497a-b9a0-d87844e267af None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:46.346734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7c8ad838-d779-497a-b9a0-d87844e267af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:46.375988 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:46.404405 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1402/2833] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:46.414079 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:46.415485 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:46.425955 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:46.472486 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a911e-82fe-438c-bf7f-58ed174e1404 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:46.476120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b770c-1567-4ce5-8caf-32c95c79ebdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:46.476704 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b770c-1567-4ce5-8caf-32c95c79ebdd service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:46.476984 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b770c-1567-4ce5-8caf-32c95c79ebdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:46.477259 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b770c-1567-4ce5-8caf-32c95c79ebdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:46.486119 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b770c-1567-4ce5-8caf-32c95c79ebdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:46.487572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b770c-1567-4ce5-8caf-32c95c79ebdd service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:46.497725 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1432/2834] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:48:47.021003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ad838-d779-497a-b9a0-d87844e267af None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:47.021003 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-7c8ad838-d779-497a-b9a0-d87844e267af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:47.031464 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ad838-d779-497a-b9a0-d87844e267af None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:47.037105 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:47.037433 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:47.037715 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:47.037989 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] PATH_INFO: `/v3/projects/302572b968444988af8811651f52d6ea` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:47.039332 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=302572b968444988af8811651f52d6ea)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:47.041337 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:47.147769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302572b968444988af8811651f52d6ea', 'request_id': 'req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.153388 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302572b968444988af8811651f52d6ea', 'request_id': 'req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.159743 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302572b968444988af8811651f52d6ea', 'request_id': 'req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.246359 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] The token cache is being invalidate because project 302572b968444988af8811651f52d6ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:48:47.246656 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.247282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.249034 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1403/2835] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:48:47 2022] DELETE /identity/v3/projects/302572b968444988af8811651f52d6ea => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:47.252971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b770c-1567-4ce5-8caf-32c95c79ebdd service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:47.263362 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b770c-1567-4ce5-8caf-32c95c79ebdd service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:47.319970 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:47.320846 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:47.321128 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:47.321404 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] PATH_INFO: `/v3/users/19b603f7577546c08ed754c2a0a354c7` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:47.322584 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19b603f7577546c08ed754c2a0a354c7)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:47.324538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:47.491777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19b603f7577546c08ed754c2a0a354c7'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.552886 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19b603f7577546c08ed754c2a0a354c7'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.553441 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19b603f7577546c08ed754c2a0a354c7'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.560625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19b603f7577546c08ed754c2a0a354c7'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.563701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6771365c-7417-470e-a2ea-d838312327a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19b603f7577546c08ed754c2a0a354c7'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:47.578392 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1433/2836] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:48:47 2022] DELETE /identity/v3/users/19b603f7577546c08ed754c2a0a354c7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:47.983700 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:47.985537 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:48.000688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e46b0b-7110-43c6-aa1a-ea5c7ef70ccc admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:48.063993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:48.064776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.065064 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.065356 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] PATH_INFO: `/v3/projects/694dc0f8bbdf44d495874455165ddc95` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.066523 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=694dc0f8bbdf44d495874455165ddc95)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:48.068443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:48.178378 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694dc0f8bbdf44d495874455165ddc95', 'request_id': 'req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:48.183848 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694dc0f8bbdf44d495874455165ddc95', 'request_id': 'req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:48.185750 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:48.189993 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694dc0f8bbdf44d495874455165ddc95', 'request_id': 'req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:48.303889 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] The token cache is being invalidate because project 694dc0f8bbdf44d495874455165ddc95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:48:48.306972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:48.306972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-2b63d5c7-fbb2-42e3-a224-85d829fc0f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:48.306972 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1404/2837] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:48:47 2022] DELETE /identity/v3/projects/694dc0f8bbdf44d495874455165ddc95 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:48.311246 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267f48c-f57c-4a26-8e29-b9dad79bbd7c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.311748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267f48c-f57c-4a26-8e29-b9dad79bbd7c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.311851 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267f48c-f57c-4a26-8e29-b9dad79bbd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.329836 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-b267f48c-f57c-4a26-8e29-b9dad79bbd7c None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:48.330990 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b267f48c-f57c-4a26-8e29-b9dad79bbd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:48.355952 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:48.391184 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1434/2838] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:48.400861 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45fd59-d9d7-4447-be5d-b481d3879fdb None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.401162 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45fd59-d9d7-4447-be5d-b481d3879fdb None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.401448 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45fd59-d9d7-4447-be5d-b481d3879fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.419929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-9d45fd59-d9d7-4447-be5d-b481d3879fdb None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:48.421138 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-9d45fd59-d9d7-4447-be5d-b481d3879fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:48.429523 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1405/2839] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:48.433599 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267f48c-f57c-4a26-8e29-b9dad79bbd7c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:48.443822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267f48c-f57c-4a26-8e29-b9dad79bbd7c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:48.448626 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e798b-680d-4315-b16e-0587282d0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:48.449279 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e798b-680d-4315-b16e-0587282d0d89 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.449564 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e798b-680d-4315-b16e-0587282d0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.449843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e798b-680d-4315-b16e-0587282d0d89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.450568 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e798b-680d-4315-b16e-0587282d0d89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:48:48.450879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e798b-680d-4315-b16e-0587282d0d89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:48.452609 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e798b-680d-4315-b16e-0587282d0d89 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:48.464917 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1435/2840] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:48:48.469102 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45fd59-d9d7-4447-be5d-b481d3879fdb None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:48.478419 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45fd59-d9d7-4447-be5d-b481d3879fdb None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:48.483121 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783489c-03b7-4605-a74c-ec3dd1437f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:48.483804 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783489c-03b7-4605-a74c-ec3dd1437f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.484093 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783489c-03b7-4605-a74c-ec3dd1437f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.484374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783489c-03b7-4605-a74c-ec3dd1437f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.485076 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783489c-03b7-4605-a74c-ec3dd1437f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:48.486977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783489c-03b7-4605-a74c-ec3dd1437f91 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:48.571918 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1406/2841] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:48:48 2022] POST /identity/v3/projects => generated 401 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:48:48.573125 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e798b-680d-4315-b16e-0587282d0d89 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:48.583522 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e798b-680d-4315-b16e-0587282d0d89 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:48.639375 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f282f-89a6-47cd-a273-3ba3afee8b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:48.640073 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f282f-89a6-47cd-a273-3ba3afee8b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.640358 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f282f-89a6-47cd-a273-3ba3afee8b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.640636 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f282f-89a6-47cd-a273-3ba3afee8b74 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.641331 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f282f-89a6-47cd-a273-3ba3afee8b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:48.643252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f282f-89a6-47cd-a273-3ba3afee8b74 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:48.746300 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1436/2842] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:48:48 2022] POST /identity/v3/users => generated 301 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:48:48.750409 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783489c-03b7-4605-a74c-ec3dd1437f91 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:48.761734 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783489c-03b7-4605-a74c-ec3dd1437f91 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:48.774080 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e935a86-1f0e-4467-abed-79b6850198f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:48.774737 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e935a86-1f0e-4467-abed-79b6850198f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.775020 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e935a86-1f0e-4467-abed-79b6850198f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.775296 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e935a86-1f0e-4467-abed-79b6850198f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.776058 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e935a86-1f0e-4467-abed-79b6850198f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:48.777658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e935a86-1f0e-4467-abed-79b6850198f9 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:48.788882 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1407/2843] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:48.793048 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f282f-89a6-47cd-a273-3ba3afee8b74 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:48.804245 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f282f-89a6-47cd-a273-3ba3afee8b74 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:48.808890 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b1b49-8720-41cb-8f06-0f9ab0db0dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:48.809544 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b1b49-8720-41cb-8f06-0f9ab0db0dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.809831 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b1b49-8720-41cb-8f06-0f9ab0db0dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.810111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b1b49-8720-41cb-8f06-0f9ab0db0dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.810843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b1b49-8720-41cb-8f06-0f9ab0db0dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:48.812478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b1b49-8720-41cb-8f06-0f9ab0db0dd1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:48.823361 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1437/2844] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:48.827460 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e935a86-1f0e-4467-abed-79b6850198f9 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:48.837427 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e935a86-1f0e-4467-abed-79b6850198f9 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:48.842042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e7fe9-a79d-4fc0-9113-fecd523fe7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:48.842916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e7fe9-a79d-4fc0-9113-fecd523fe7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.843198 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e7fe9-a79d-4fc0-9113-fecd523fe7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.843473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e7fe9-a79d-4fc0-9113-fecd523fe7f1 admin admin] PATH_INFO: `/v3/projects/2a5ab8267d2c4d9c8851c4957a0a4e69/users/9107cbe3a10144c884566db2fbbb2248/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.857620 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e7fe9-a79d-4fc0-9113-fecd523fe7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5ab8267d2c4d9c8851c4957a0a4e69, user_id=9107cbe3a10144c884566db2fbbb2248, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:48.860305 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e7fe9-a79d-4fc0-9113-fecd523fe7f1 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:48.961199 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1408/2845] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:48:48 2022] PUT /identity/v3/projects/2a5ab8267d2c4d9c8851c4957a0a4e69/users/9107cbe3a10144c884566db2fbbb2248/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:48.971435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885a68c-3c92-4bfb-a188-0a1eaa0b363c None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:48.971986 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885a68c-3c92-4bfb-a188-0a1eaa0b363c None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:48.972121 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885a68c-3c92-4bfb-a188-0a1eaa0b363c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:48.998971 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-9885a68c-3c92-4bfb-a188-0a1eaa0b363c None None] MFA Rules not processed for user `9107cbe3a10144c884566db2fbbb2248`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:49.000218 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9885a68c-3c92-4bfb-a188-0a1eaa0b363c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:49.063540 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1438/2846] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:49.067996 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e7fe9-a79d-4fc0-9113-fecd523fe7f1 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:49.129857 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e7fe9-a79d-4fc0-9113-fecd523fe7f1 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:49.134452 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1700c-b136-4d66-8f70-b6125a2b8fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:49.135107 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1700c-b136-4d66-8f70-b6125a2b8fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:49.135392 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1700c-b136-4d66-8f70-b6125a2b8fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:49.135706 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1700c-b136-4d66-8f70-b6125a2b8fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:49.136413 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1700c-b136-4d66-8f70-b6125a2b8fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:49.138313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1700c-b136-4d66-8f70-b6125a2b8fd7 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:49.277954 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885a68c-3c92-4bfb-a188-0a1eaa0b363c None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:49.285984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1409/2847] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:48:49 2022] POST /identity/v3/projects => generated 399 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:48:49.295734 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885a68c-3c92-4bfb-a188-0a1eaa0b363c None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:49.316563 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:49.353493 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f229f-adbf-4315-9c98-d538db27573b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:49.354168 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f229f-adbf-4315-9c98-d538db27573b admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:49.354710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f229f-adbf-4315-9c98-d538db27573b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:49.354979 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f229f-adbf-4315-9c98-d538db27573b admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:49.355906 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f229f-adbf-4315-9c98-d538db27573b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:49.358380 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f229f-adbf-4315-9c98-d538db27573b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:49.440276 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1439/2848] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:48:49 2022] POST /identity/v3/users => generated 306 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:48:49.444243 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1700c-b136-4d66-8f70-b6125a2b8fd7 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:49.456443 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1700c-b136-4d66-8f70-b6125a2b8fd7 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:49.461887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ba878-e5a1-457e-9c0f-1df9de6361a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:49.464263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ba878-e5a1-457e-9c0f-1df9de6361a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:49.464263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ba878-e5a1-457e-9c0f-1df9de6361a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:49.464263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ba878-e5a1-457e-9c0f-1df9de6361a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:49.464263 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ba878-e5a1-457e-9c0f-1df9de6361a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:49.465788 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ba878-e5a1-457e-9c0f-1df9de6361a8 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:49.478262 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1410/2849] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:49.482641 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f229f-adbf-4315-9c98-d538db27573b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:49.492922 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f229f-adbf-4315-9c98-d538db27573b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:49.497652 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700bf94-0c86-413d-9419-b49de7ccc814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:49.498532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700bf94-0c86-413d-9419-b49de7ccc814 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:49.498816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700bf94-0c86-413d-9419-b49de7ccc814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:49.499089 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700bf94-0c86-413d-9419-b49de7ccc814 admin admin] PATH_INFO: `/v3/projects/7120d65ef2a2494fafee3438df9418e1/users/c6945ee1c8734aa1895de9981236a477/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:49.513886 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700bf94-0c86-413d-9419-b49de7ccc814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7120d65ef2a2494fafee3438df9418e1, user_id=c6945ee1c8734aa1895de9981236a477, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:49.516606 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700bf94-0c86-413d-9419-b49de7ccc814 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:49.587035 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1440/2850] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:48:49 2022] PUT /identity/v3/projects/7120d65ef2a2494fafee3438df9418e1/users/c6945ee1c8734aa1895de9981236a477/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:48:49.591113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ba878-e5a1-457e-9c0f-1df9de6361a8 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:49.601903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ba878-e5a1-457e-9c0f-1df9de6361a8 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:49.657353 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533090d-ed2e-4ba7-955c-c5cbdc8e2cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:49.658021 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533090d-ed2e-4ba7-955c-c5cbdc8e2cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:49.658307 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533090d-ed2e-4ba7-955c-c5cbdc8e2cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:49.658587 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533090d-ed2e-4ba7-955c-c5cbdc8e2cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:49.659311 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533090d-ed2e-4ba7-955c-c5cbdc8e2cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:49.660960 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533090d-ed2e-4ba7-955c-c5cbdc8e2cb3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:49.662323 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:49.671282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1411/2851] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:49.675625 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700bf94-0c86-413d-9419-b49de7ccc814 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:49.686083 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700bf94-0c86-413d-9419-b49de7ccc814 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:49.690876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e638c3-f2a9-478f-afdd-3d9b4c70eef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:49.692215 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e638c3-f2a9-478f-afdd-3d9b4c70eef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:49.692499 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e638c3-f2a9-478f-afdd-3d9b4c70eef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:49.692774 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e638c3-f2a9-478f-afdd-3d9b4c70eef8 admin admin] PATH_INFO: `/v3/projects/7120d65ef2a2494fafee3438df9418e1/users/c6945ee1c8734aa1895de9981236a477/roles/03b724e03f704f2ab338b64025a3db6d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:49.694837 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e638c3-f2a9-478f-afdd-3d9b4c70eef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7120d65ef2a2494fafee3438df9418e1, user_id=c6945ee1c8734aa1895de9981236a477, role_id=03b724e03f704f2ab338b64025a3db6d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:49.697458 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e638c3-f2a9-478f-afdd-3d9b4c70eef8 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:49.708469 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1441/2852] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:48:49 2022] PUT /identity/v3/projects/7120d65ef2a2494fafee3438df9418e1/users/c6945ee1c8734aa1895de9981236a477/roles/03b724e03f704f2ab338b64025a3db6d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:49.719328 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dec7d4-3167-4033-808e-c22b44f15758 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:49.719648 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dec7d4-3167-4033-808e-c22b44f15758 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:49.719954 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dec7d4-3167-4033-808e-c22b44f15758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:49.745073 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-59dec7d4-3167-4033-808e-c22b44f15758 None None] MFA Rules not processed for user `c6945ee1c8734aa1895de9981236a477`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:49.746338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-59dec7d4-3167-4033-808e-c22b44f15758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:49.812406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1412/2853] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:49 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:49.823825 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e638c3-f2a9-478f-afdd-3d9b4c70eef8 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:49.825498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-98e638c3-f2a9-478f-afdd-3d9b4c70eef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:49.880950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e638c3-f2a9-478f-afdd-3d9b4c70eef8 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:49.884616 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984dba-7030-4cfc-85c6-de990778d2aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:49.885250 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984dba-7030-4cfc-85c6-de990778d2aa service cinder] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:49.885531 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984dba-7030-4cfc-85c6-de990778d2aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:49.885803 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984dba-7030-4cfc-85c6-de990778d2aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:49.895581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984dba-7030-4cfc-85c6-de990778d2aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:49.897026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984dba-7030-4cfc-85c6-de990778d2aa service cinder] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:49.906822 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1442/2854] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:48:49 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:48:50.120565 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dec7d4-3167-4033-808e-c22b44f15758 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:50.133166 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dec7d4-3167-4033-808e-c22b44f15758 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:50.136839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13beca-4175-41ad-9d62-cbceddc6591c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:50.139518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13beca-4175-41ad-9d62-cbceddc6591c service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:50.139518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13beca-4175-41ad-9d62-cbceddc6591c service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:50.139518 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13beca-4175-41ad-9d62-cbceddc6591c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:50.195858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13beca-4175-41ad-9d62-cbceddc6591c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:50.197322 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13beca-4175-41ad-9d62-cbceddc6591c service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:50.206249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1413/2855] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:48:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:48:55.004600 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984dba-7030-4cfc-85c6-de990778d2aa service cinder] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:55.067651 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984dba-7030-4cfc-85c6-de990778d2aa service cinder] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:55.072252 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:55.073120 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:55.073397 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:55.073674 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] PATH_INFO: `/v3/users/9107cbe3a10144c884566db2fbbb2248` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:55.074853 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9107cbe3a10144c884566db2fbbb2248)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:55.076802 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:55.098649 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:55.286830 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9107cbe3a10144c884566db2fbbb2248'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:55.358208 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9107cbe3a10144c884566db2fbbb2248'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:55.358842 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9107cbe3a10144c884566db2fbbb2248'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:55.365974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9107cbe3a10144c884566db2fbbb2248'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:55.366597 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9107cbe3a10144c884566db2fbbb2248'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:55.383426 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1443/2856] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:48:55 2022] DELETE /identity/v3/users/9107cbe3a10144c884566db2fbbb2248 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:55.387915 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3b281-c9c5-4465-86f5-12b038adc66a None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:55.388391 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3b281-c9c5-4465-86f5-12b038adc66a None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:55.395906 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3b281-c9c5-4465-86f5-12b038adc66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:55.405714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-24d3b281-c9c5-4465-86f5-12b038adc66a None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:55.406925 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-24d3b281-c9c5-4465-86f5-12b038adc66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:55.465512 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1414/2857] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:55.475742 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:55.477116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:55.532869 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617310b-3966-4555-a1c4-3fbb0301a00b admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:55.537111 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d61a0-68f0-4e79-b03d-af10051b7c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:55.537749 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d61a0-68f0-4e79-b03d-af10051b7c05 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:55.538033 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d61a0-68f0-4e79-b03d-af10051b7c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:55.538328 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d61a0-68f0-4e79-b03d-af10051b7c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:55.547472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d61a0-68f0-4e79-b03d-af10051b7c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:55.548921 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d61a0-68f0-4e79-b03d-af10051b7c05 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:55.558291 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1444/2858] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:48:55.953977 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3b281-c9c5-4465-86f5-12b038adc66a None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:55.955451 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-24d3b281-c9c5-4465-86f5-12b038adc66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:55.968142 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3b281-c9c5-4465-86f5-12b038adc66a None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:55.972744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:55.974015 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:55.974299 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:55.974578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] PATH_INFO: `/v3/projects/2a5ab8267d2c4d9c8851c4957a0a4e69` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:55.975821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5ab8267d2c4d9c8851c4957a0a4e69)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:55.977731 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:55.994517 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=100 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:56.084426 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5ab8267d2c4d9c8851c4957a0a4e69', 'request_id': 'req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.090642 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5ab8267d2c4d9c8851c4957a0a4e69', 'request_id': 'req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.097505 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5ab8267d2c4d9c8851c4957a0a4e69', 'request_id': 'req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.168929 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] The token cache is being invalidate because project 2a5ab8267d2c4d9c8851c4957a0a4e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:48:56.169223 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.169842 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.171489 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1415/2859] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:48:55 2022] DELETE /identity/v3/projects/2a5ab8267d2c4d9c8851c4957a0a4e69 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:56.175718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d61a0-68f0-4e79-b03d-af10051b7c05 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:56.185816 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d61a0-68f0-4e79-b03d-af10051b7c05 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:56.241046 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:56.241912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:56.242196 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:56.242472 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] PATH_INFO: `/v3/users/c6945ee1c8734aa1895de9981236a477` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:56.243690 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6945ee1c8734aa1895de9981236a477)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:56.245620 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:56.449366 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:56.451874 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6945ee1c8734aa1895de9981236a477'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.527132 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6945ee1c8734aa1895de9981236a477'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.527788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6945ee1c8734aa1895de9981236a477'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.534733 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6945ee1c8734aa1895de9981236a477'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.536012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-a4288ec1-36ba-401c-97c7-e16a040b174c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6945ee1c8734aa1895de9981236a477'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:56.553581 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1445/2860] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:48:56 2022] DELETE /identity/v3/users/c6945ee1c8734aa1895de9981236a477 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:56.960455 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:56.964819 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:56.976189 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d64cf-7cc2-44cc-ab40-8ee72ac5fba5 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:57.033366 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:57.034233 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.034526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.034793 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] PATH_INFO: `/v3/projects/7120d65ef2a2494fafee3438df9418e1` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.036633 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7120d65ef2a2494fafee3438df9418e1)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:57.038542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:57.133339 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7120d65ef2a2494fafee3438df9418e1', 'request_id': 'req-410498c8-8cd8-44e6-83b8-65e21ff7d403'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:57.138950 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7120d65ef2a2494fafee3438df9418e1', 'request_id': 'req-410498c8-8cd8-44e6-83b8-65e21ff7d403'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:57.145851 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7120d65ef2a2494fafee3438df9418e1', 'request_id': 'req-410498c8-8cd8-44e6-83b8-65e21ff7d403'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:57.198018 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] The token cache is being invalidate because project 7120d65ef2a2494fafee3438df9418e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:48:57.198306 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:57.198932 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-410498c8-8cd8-44e6-83b8-65e21ff7d403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:48:57.200549 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1416/2861] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:48:56 2022] DELETE /identity/v3/projects/7120d65ef2a2494fafee3438df9418e1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:48:57.205675 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3941b89-19fe-4963-9fba-60f5affb6b08 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.206139 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3941b89-19fe-4963-9fba-60f5affb6b08 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.206256 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3941b89-19fe-4963-9fba-60f5affb6b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.224763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-c3941b89-19fe-4963-9fba-60f5affb6b08 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:57.225301 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-c3941b89-19fe-4963-9fba-60f5affb6b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:57.283590 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1446/2862] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:57.291290 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cb5ba-80ff-4447-b6ab-fd7c8e37f481 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.291622 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cb5ba-80ff-4447-b6ab-fd7c8e37f481 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.291930 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cb5ba-80ff-4447-b6ab-fd7c8e37f481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.311084 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-121cb5ba-80ff-4447-b6ab-fd7c8e37f481 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:57.312313 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-121cb5ba-80ff-4447-b6ab-fd7c8e37f481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:57.320661 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1417/2863] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:57.324683 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3941b89-19fe-4963-9fba-60f5affb6b08 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:57.334912 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3941b89-19fe-4963-9fba-60f5affb6b08 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:57.339387 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12f890-d0a6-44dd-ad85-b7f8218c2c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:57.340071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12f890-d0a6-44dd-ad85-b7f8218c2c8d admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.340351 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12f890-d0a6-44dd-ad85-b7f8218c2c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.340631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12f890-d0a6-44dd-ad85-b7f8218c2c8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.341341 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12f890-d0a6-44dd-ad85-b7f8218c2c8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:48:57.341642 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12f890-d0a6-44dd-ad85-b7f8218c2c8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:57.343254 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12f890-d0a6-44dd-ad85-b7f8218c2c8d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:57.354577 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1447/2864] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:48:57.358580 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cb5ba-80ff-4447-b6ab-fd7c8e37f481 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:57.367542 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cb5ba-80ff-4447-b6ab-fd7c8e37f481 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:57.372152 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6cd59-897a-4745-b943-818749ac5f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:57.372790 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6cd59-897a-4745-b943-818749ac5f65 admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.373070 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6cd59-897a-4745-b943-818749ac5f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.373341 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6cd59-897a-4745-b943-818749ac5f65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.374036 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6cd59-897a-4745-b943-818749ac5f65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:57.375931 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6cd59-897a-4745-b943-818749ac5f65 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:57.473602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1418/2865] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:48:57 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 09:48:57.477880 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12f890-d0a6-44dd-ad85-b7f8218c2c8d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:57.487879 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12f890-d0a6-44dd-ad85-b7f8218c2c8d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:57.543875 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ed93c-1f6f-4ddc-9b87-416c3d5ef2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:57.544542 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed93c-1f6f-4ddc-9b87-416c3d5ef2fc admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.544828 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed93c-1f6f-4ddc-9b87-416c3d5ef2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.545109 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed93c-1f6f-4ddc-9b87-416c3d5ef2fc admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.545794 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ed93c-1f6f-4ddc-9b87-416c3d5ef2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:57.548002 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ed93c-1f6f-4ddc-9b87-416c3d5ef2fc admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:57.666186 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1448/2866] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:48:57 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:48:57.670236 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6cd59-897a-4745-b943-818749ac5f65 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:57.677638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:57.681430 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6cd59-897a-4745-b943-818749ac5f65 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:57.686179 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d96e7-a9e8-4595-b5b4-4c09ebaf71d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:57.686838 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d96e7-a9e8-4595-b5b4-4c09ebaf71d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.687120 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d96e7-a9e8-4595-b5b4-4c09ebaf71d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.687396 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d96e7-a9e8-4595-b5b4-4c09ebaf71d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.688171 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d96e7-a9e8-4595-b5b4-4c09ebaf71d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:57.689744 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d96e7-a9e8-4595-b5b4-4c09ebaf71d8 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:57.701047 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1419/2867] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:48:57.705801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ed93c-1f6f-4ddc-9b87-416c3d5ef2fc admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:57.715086 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ed93c-1f6f-4ddc-9b87-416c3d5ef2fc admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:57.719688 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1dc95c-ed66-4bfc-9a5c-2f58e43d5e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:57.720335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dc95c-ed66-4bfc-9a5c-2f58e43d5e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.720619 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dc95c-ed66-4bfc-9a5c-2f58e43d5e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.720893 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dc95c-ed66-4bfc-9a5c-2f58e43d5e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.721631 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dc95c-ed66-4bfc-9a5c-2f58e43d5e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:57.723211 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dc95c-ed66-4bfc-9a5c-2f58e43d5e0e admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:57.733022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1449/2868] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 09:48:57.737277 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d96e7-a9e8-4595-b5b4-4c09ebaf71d8 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:57.747979 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d96e7-a9e8-4595-b5b4-4c09ebaf71d8 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:57.754776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3060dab-fd3a-482c-885e-c08be6e1dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:57.754776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3060dab-fd3a-482c-885e-c08be6e1dee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.754776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3060dab-fd3a-482c-885e-c08be6e1dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.754776 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3060dab-fd3a-482c-885e-c08be6e1dee3 admin admin] PATH_INFO: `/v3/projects/7df4073fd39f4fe893204c3327082d65/users/436ab1af0e1243c0b3c5c81f4e7830a5/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.768963 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3060dab-fd3a-482c-885e-c08be6e1dee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df4073fd39f4fe893204c3327082d65, user_id=436ab1af0e1243c0b3c5c81f4e7830a5, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:57.771703 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3060dab-fd3a-482c-885e-c08be6e1dee3 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:57.846526 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1420/2869] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:48:57 2022] PUT /identity/v3/projects/7df4073fd39f4fe893204c3327082d65/users/436ab1af0e1243c0b3c5c81f4e7830a5/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:48:57.856718 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cc951-f55d-4c17-bc97-2ad57e1c7503 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:57.857113 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cc951-f55d-4c17-bc97-2ad57e1c7503 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:57.857394 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cc951-f55d-4c17-bc97-2ad57e1c7503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:57.884876 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a95cc951-f55d-4c17-bc97-2ad57e1c7503 None None] MFA Rules not processed for user `436ab1af0e1243c0b3c5c81f4e7830a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:48:57.886123 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a95cc951-f55d-4c17-bc97-2ad57e1c7503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:57.898543 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:48:57.951131 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1450/2870] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:48:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:48:57.961763 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3060dab-fd3a-482c-885e-c08be6e1dee3 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:48:57.964193 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-f3060dab-fd3a-482c-885e-c08be6e1dee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:48:58.020055 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3060dab-fd3a-482c-885e-c08be6e1dee3 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:48:58.023594 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7783b-1ddd-4bf3-b700-b274f5c90d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:48:58.024234 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7783b-1ddd-4bf3-b700-b274f5c90d0a service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:48:58.024515 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7783b-1ddd-4bf3-b700-b274f5c90d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:48:58.024792 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7783b-1ddd-4bf3-b700-b274f5c90d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:48:58.033046 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7783b-1ddd-4bf3-b700-b274f5c90d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:48:58.034473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7783b-1ddd-4bf3-b700-b274f5c90d0a service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:48:58.043702 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1421/2871] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:48:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:49:07.886572 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cc951-f55d-4c17-bc97-2ad57e1c7503 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:07.947176 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cc951-f55d-4c17-bc97-2ad57e1c7503 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:07.951672 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:07.952532 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:07.952808 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:07.953078 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] PATH_INFO: `/v3/users/436ab1af0e1243c0b3c5c81f4e7830a5` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:07.954239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=436ab1af0e1243c0b3c5c81f4e7830a5)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:07.956188 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:08.039733 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:49:08.135474 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436ab1af0e1243c0b3c5c81f4e7830a5'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:08.201064 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436ab1af0e1243c0b3c5c81f4e7830a5'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:08.201676 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436ab1af0e1243c0b3c5c81f4e7830a5'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:08.208867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436ab1af0e1243c0b3c5c81f4e7830a5'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:08.209562 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436ab1af0e1243c0b3c5c81f4e7830a5'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:08.227021 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1451/2872] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:49:07 2022] DELETE /identity/v3/users/436ab1af0e1243c0b3c5c81f4e7830a5 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:49:08.231488 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c91e-d127-40e3-8593-a69ff546b27e None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:08.231821 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c91e-d127-40e3-8593-a69ff546b27e None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:08.232802 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825c91e-d127-40e3-8593-a69ff546b27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:08.249361 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-b825c91e-d127-40e3-8593-a69ff546b27e None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:49:08.250578 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b825c91e-d127-40e3-8593-a69ff546b27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:08.308984 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1422/2873] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:49:08.319318 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:08.320766 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:08.359639 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:49:08.376699 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9339d0a-b00d-40de-82ab-bb50750042b0 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:08.380998 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61696b5-b6b5-428d-8677-7b67cf36f731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:08.381637 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61696b5-b6b5-428d-8677-7b67cf36f731 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:08.381918 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61696b5-b6b5-428d-8677-7b67cf36f731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:08.382191 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61696b5-b6b5-428d-8677-7b67cf36f731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:08.391335 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61696b5-b6b5-428d-8677-7b67cf36f731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:08.392763 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61696b5-b6b5-428d-8677-7b67cf36f731 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:08.402512 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1452/2874] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:49:08.796727 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c91e-d127-40e3-8593-a69ff546b27e None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:08.798444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-b825c91e-d127-40e3-8593-a69ff546b27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:08.810688 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825c91e-d127-40e3-8593-a69ff546b27e None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:08.815858 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:08.816805 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:08.817091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:08.817380 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] PATH_INFO: `/v3/projects/7df4073fd39f4fe893204c3327082d65` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:08.818592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df4073fd39f4fe893204c3327082d65)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:08.820546 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:08.916197 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df4073fd39f4fe893204c3327082d65', 'request_id': 'req-20b0bd53-92e7-4858-9967-7f75bc24b755'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:08.921592 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df4073fd39f4fe893204c3327082d65', 'request_id': 'req-20b0bd53-92e7-4858-9967-7f75bc24b755'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:08.927962 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df4073fd39f4fe893204c3327082d65', 'request_id': 'req-20b0bd53-92e7-4858-9967-7f75bc24b755'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:09.000442 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] The token cache is being invalidate because project 7df4073fd39f4fe893204c3327082d65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:49:09.000729 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:09.001338 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-20b0bd53-92e7-4858-9967-7f75bc24b755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:09.003013 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1423/2875] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:49:08 2022] DELETE /identity/v3/projects/7df4073fd39f4fe893204c3327082d65 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 09:49:09.007987 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861257db-0a79-49bc-ab07-8d2a707eabc4 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.008467 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861257db-0a79-49bc-ab07-8d2a707eabc4 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.008583 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861257db-0a79-49bc-ab07-8d2a707eabc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.026457 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-861257db-0a79-49bc-ab07-8d2a707eabc4 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:49:09.027701 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-861257db-0a79-49bc-ab07-8d2a707eabc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:09.085677 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1453/2876] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:49:09.094885 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4bb9a-96bf-4243-b09f-eb2dbfbee1e3 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.095374 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4bb9a-96bf-4243-b09f-eb2dbfbee1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.095480 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4bb9a-96bf-4243-b09f-eb2dbfbee1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.114385 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-22e4bb9a-96bf-4243-b09f-eb2dbfbee1e3 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:49:09.115633 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-22e4bb9a-96bf-4243-b09f-eb2dbfbee1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:09.123875 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1424/2877] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:49:09.127983 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861257db-0a79-49bc-ab07-8d2a707eabc4 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:09.138266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861257db-0a79-49bc-ab07-8d2a707eabc4 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:09.142850 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963994d-cb33-4aad-9ce9-e7af54527bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:09.143507 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963994d-cb33-4aad-9ce9-e7af54527bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.143829 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963994d-cb33-4aad-9ce9-e7af54527bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.144107 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963994d-cb33-4aad-9ce9-e7af54527bf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.145406 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963994d-cb33-4aad-9ce9-e7af54527bf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80345) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 09:49:09.145713 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963994d-cb33-4aad-9ce9-e7af54527bf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:09.147305 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963994d-cb33-4aad-9ce9-e7af54527bf0 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:09.158809 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1454/2878] 10.222.0.64 () {58 vars in 1152 bytes} [Mon Jun 27 09:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 09:49:09.163111 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4bb9a-96bf-4243-b09f-eb2dbfbee1e3 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:09.173124 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4bb9a-96bf-4243-b09f-eb2dbfbee1e3 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:09.178432 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540920fc-d3dc-4ff7-a139-122ecce3f87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:09.179301 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540920fc-d3dc-4ff7-a139-122ecce3f87a admin admin] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.179602 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540920fc-d3dc-4ff7-a139-122ecce3f87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.179887 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540920fc-d3dc-4ff7-a139-122ecce3f87a admin admin] PATH_INFO: `/v3/projects` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.180587 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540920fc-d3dc-4ff7-a139-122ecce3f87a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:09.182504 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540920fc-d3dc-4ff7-a139-122ecce3f87a admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:09.267828 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1425/2879] 10.222.0.64 () {60 vars in 1152 bytes} [Mon Jun 27 09:49:09 2022] POST /identity/v3/projects => generated 399 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:49:09.272582 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963994d-cb33-4aad-9ce9-e7af54527bf0 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:09.283095 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963994d-cb33-4aad-9ce9-e7af54527bf0 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:09.345498 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffed82-9a3f-48dd-8a93-4f50dd93d9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:09.346169 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffed82-9a3f-48dd-8a93-4f50dd93d9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.346452 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffed82-9a3f-48dd-8a93-4f50dd93d9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.346724 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffed82-9a3f-48dd-8a93-4f50dd93d9f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.347446 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffed82-9a3f-48dd-8a93-4f50dd93d9f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:09.349376 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffed82-9a3f-48dd-8a93-4f50dd93d9f5 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:09.428129 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1455/2880] 10.222.0.64 () {60 vars in 1143 bytes} [Mon Jun 27 09:49:09 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 09:49:09.432066 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540920fc-d3dc-4ff7-a139-122ecce3f87a admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:09.442921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540920fc-d3dc-4ff7-a139-122ecce3f87a admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:09.447571 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d9914-e800-4dbc-a51b-64df0d6339fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:09.448283 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d9914-e800-4dbc-a51b-64df0d6339fc admin admin] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.448557 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d9914-e800-4dbc-a51b-64df0d6339fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.448840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d9914-e800-4dbc-a51b-64df0d6339fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.449568 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d9914-e800-4dbc-a51b-64df0d6339fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:09.451221 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d9914-e800-4dbc-a51b-64df0d6339fc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:09.462334 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1426/2881] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:49:09.469614 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffed82-9a3f-48dd-8a93-4f50dd93d9f5 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:09.479804 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffed82-9a3f-48dd-8a93-4f50dd93d9f5 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:09.484843 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14330a0e-5c7a-44a1-bc22-73f071d21559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:09.485504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14330a0e-5c7a-44a1-bc22-73f071d21559 admin admin] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.485788 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14330a0e-5c7a-44a1-bc22-73f071d21559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.486065 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14330a0e-5c7a-44a1-bc22-73f071d21559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.486790 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14330a0e-5c7a-44a1-bc22-73f071d21559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:09.488943 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14330a0e-5c7a-44a1-bc22-73f071d21559 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:09.499266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1456/2882] 10.222.0.64 () {58 vars in 1121 bytes} [Mon Jun 27 09:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 09:49:09.505287 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d9914-e800-4dbc-a51b-64df0d6339fc admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:09.516435 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d9914-e800-4dbc-a51b-64df0d6339fc admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:09.521358 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c726fb1-e992-4dfe-9250-a2833bf190cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:09.522249 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c726fb1-e992-4dfe-9250-a2833bf190cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.522533 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c726fb1-e992-4dfe-9250-a2833bf190cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.522840 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c726fb1-e992-4dfe-9250-a2833bf190cc admin admin] PATH_INFO: `/v3/projects/049fe8b4f0fa4aa1a78349d3c4dbe7db/users/da645d9ca4b54c409a96ad987a37563d/roles/44cbd523412b44e3a591b3111fb50a71` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.538395 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c726fb1-e992-4dfe-9250-a2833bf190cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=049fe8b4f0fa4aa1a78349d3c4dbe7db, user_id=da645d9ca4b54c409a96ad987a37563d, role_id=44cbd523412b44e3a591b3111fb50a71)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:09.541159 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c726fb1-e992-4dfe-9250-a2833bf190cc admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:09.659173 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1427/2883] 10.222.0.64 () {60 vars in 1482 bytes} [Mon Jun 27 09:49:09 2022] PUT /identity/v3/projects/049fe8b4f0fa4aa1a78349d3c4dbe7db/users/da645d9ca4b54c409a96ad987a37563d/roles/44cbd523412b44e3a591b3111fb50a71 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:49:09.669319 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7d8ab-2753-48bf-bc69-95bd61194138 None None] REQUEST_METHOD: `POST` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.669693 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7d8ab-2753-48bf-bc69-95bd61194138 None None] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.669974 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7d8ab-2753-48bf-bc69-95bd61194138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.696883 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.auth.core [None req-a5a7d8ab-2753-48bf-bc69-95bd61194138 None None] MFA Rules not processed for user `da645d9ca4b54c409a96ad987a37563d`. Rule list: `[]` (Enabled: `True`). {{(pid=80345) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:49:09.698001 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-a5a7d8ab-2753-48bf-bc69-95bd61194138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:09.729018 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:49:09.760401 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1457/2884] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:49:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 09:49:09.771444 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c726fb1-e992-4dfe-9250-a2833bf190cc admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:09.773130 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-5c726fb1-e992-4dfe-9250-a2833bf190cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:09.795834 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:49:09.849251 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c726fb1-e992-4dfe-9250-a2833bf190cc admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:09.852951 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dae7eb-3b71-4b19-9eb4-57527027317c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c8abb981d1b4c82a43b5a3372f98398', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:09.853593 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dae7eb-3b71-4b19-9eb4-57527027317c service cinder] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:09.853881 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dae7eb-3b71-4b19-9eb4-57527027317c service cinder] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:09.854158 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dae7eb-3b71-4b19-9eb4-57527027317c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:09.862327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dae7eb-3b71-4b19-9eb4-57527027317c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:09.864282 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dae7eb-3b71-4b19-9eb4-57527027317c service cinder] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:09.873425 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1428/2885] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:49:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:49:18.780755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7d8ab-2753-48bf-bc69-95bd61194138 None None] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:18.841556 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7d8ab-2753-48bf-bc69-95bd61194138 None None] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:18.845941 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:18.846797 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:18.847074 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:18.847347 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] PATH_INFO: `/v3/users/da645d9ca4b54c409a96ad987a37563d` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:18.849052 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] RBAC: Authorizing `identity:delete_user(user_id=da645d9ca4b54c409a96ad987a37563d)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:18.851024 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:19.039841 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da645d9ca4b54c409a96ad987a37563d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.106436 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da645d9ca4b54c409a96ad987a37563d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.107022 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da645d9ca4b54c409a96ad987a37563d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.115036 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da645d9ca4b54c409a96ad987a37563d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.115710 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da645d9ca4b54c409a96ad987a37563d'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.132948 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1458/2886] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:49:18 2022] DELETE /identity/v3/users/da645d9ca4b54c409a96ad987a37563d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:49:19.137476 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa235b19-1404-4812-ac90-c7dacae63e50 None None] REQUEST_METHOD: `POST` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:19.137971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa235b19-1404-4812-ac90-c7dacae63e50 None None] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:19.138097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa235b19-1404-4812-ac90-c7dacae63e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:19.155050 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.auth.core [None req-aa235b19-1404-4812-ac90-c7dacae63e50 None None] MFA Rules not processed for user `e1fa6647d77b40cd8875b5b59ec1508d`. Rule list: `[]` (Enabled: `True`). {{(pid=80344) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 09:49:19.156264 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-aa235b19-1404-4812-ac90-c7dacae63e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:19.215349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1429/2887] 10.222.0.64 () {58 vars in 957 bytes} [Mon Jun 27 09:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 09:49:19.225355 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:19.226820 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:19.283569 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd59e41-18f9-445f-90b0-b52ab162e08d admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:19.288042 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44cf90-8b0d-4ce4-847b-9f50b58c9592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:19.288689 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44cf90-8b0d-4ce4-847b-9f50b58c9592 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:19.288966 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44cf90-8b0d-4ce4-847b-9f50b58c9592 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:19.289239 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44cf90-8b0d-4ce4-847b-9f50b58c9592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:19.296435 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:49:19.299012 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44cf90-8b0d-4ce4-847b-9f50b58c9592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:19.300504 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44cf90-8b0d-4ce4-847b-9f50b58c9592 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:19.309796 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1459/2888] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:49:19.684991 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa235b19-1404-4812-ac90-c7dacae63e50 None None] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:49:19.689652 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-aa235b19-1404-4812-ac90-c7dacae63e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:49:19.698647 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa235b19-1404-4812-ac90-c7dacae63e50 None None] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:49:19.703886 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:49:19.704769 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:49:19.705054 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:49:19.705329 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] PATH_INFO: `/v3/projects/049fe8b4f0fa4aa1a78349d3c4dbe7db` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:49:19.706561 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=049fe8b4f0fa4aa1a78349d3c4dbe7db)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:49:19.713241 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:49:19.803403 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '049fe8b4f0fa4aa1a78349d3c4dbe7db', 'request_id': 'req-09a36ed3-1c84-4e5b-8034-9eca0c541fde'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.808921 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '049fe8b4f0fa4aa1a78349d3c4dbe7db', 'request_id': 'req-09a36ed3-1c84-4e5b-8034-9eca0c541fde'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.815118 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '049fe8b4f0fa4aa1a78349d3c4dbe7db', 'request_id': 'req-09a36ed3-1c84-4e5b-8034-9eca0c541fde'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.903324 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] The token cache is being invalidate because project 049fe8b4f0fa4aa1a78349d3c4dbe7db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:49:19.903655 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.904279 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:49:19.905839 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1430/2889] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:49:19 2022] DELETE /identity/v3/projects/049fe8b4f0fa4aa1a78349d3c4dbe7db => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:49:29.466183 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=2 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:49:29.900202 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:50:05.393747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44cf90-8b0d-4ce4-847b-9f50b58c9592 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:50:05.395235 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-2b44cf90-8b0d-4ce4-847b-9f50b58c9592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:50:05.484624 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44cf90-8b0d-4ce4-847b-9f50b58c9592 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:50:05.543855 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:50:05.544503 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:50:05.544786 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:50:05.545063 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:50:05.546867 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:50:05.558801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:50:05.560399 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:50:05.570174 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1460/2890] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:50:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:50:10.085710 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:50:10.087136 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:50:10.146178 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a36ed3-1c84-4e5b-8034-9eca0c541fde admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:50:10.149669 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dcc26-8fc2-44a0-b32a-8a79dcb6ef09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:50:10.150304 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dcc26-8fc2-44a0-b32a-8a79dcb6ef09 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:50:10.150584 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dcc26-8fc2-44a0-b32a-8a79dcb6ef09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:50:10.150855 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dcc26-8fc2-44a0-b32a-8a79dcb6ef09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:50:10.152608 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-282dcc26-8fc2-44a0-b32a-8a79dcb6ef09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:50:10.162280 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dcc26-8fc2-44a0-b32a-8a79dcb6ef09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:50:10.163718 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dcc26-8fc2-44a0-b32a-8a79dcb6ef09 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:50:10.171550 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1431/2891] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:50:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:50:10.416680 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:50:10.418230 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:50:10.430399 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171382f-7401-4d79-bf69-0147b8ff1342 service placement] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:50:10.433976 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287af14-7ed5-41f9-9c26-0d99da922868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:50:10.434709 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287af14-7ed5-41f9-9c26-0d99da922868 service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:50:10.434978 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287af14-7ed5-41f9-9c26-0d99da922868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:50:10.435377 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287af14-7ed5-41f9-9c26-0d99da922868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:50:10.437277 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-9287af14-7ed5-41f9-9c26-0d99da922868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:50:10.447266 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287af14-7ed5-41f9-9c26-0d99da922868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:50:10.448849 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287af14-7ed5-41f9-9c26-0d99da922868 service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:50:10.457801 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1461/2892] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:50:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:50:10.553792 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dcc26-8fc2-44a0-b32a-8a79dcb6ef09 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:50:10.562449 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dcc26-8fc2-44a0-b32a-8a79dcb6ef09 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:50:10.566030 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce43aaa-a010-43e8-8cbf-14dd947ee202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:50:10.566658 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce43aaa-a010-43e8-8cbf-14dd947ee202 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:50:10.566942 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce43aaa-a010-43e8-8cbf-14dd947ee202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:50:10.567213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce43aaa-a010-43e8-8cbf-14dd947ee202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:50:10.568955 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-0ce43aaa-a010-43e8-8cbf-14dd947ee202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:50:10.656273 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce43aaa-a010-43e8-8cbf-14dd947ee202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:50:10.657726 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce43aaa-a010-43e8-8cbf-14dd947ee202 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:50:10.677764 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1432/2893] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:50:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:50:20.453478 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=73 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:50:20.675091 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=72 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:51:40.490061 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287af14-7ed5-41f9-9c26-0d99da922868 service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:51:40.499348 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287af14-7ed5-41f9-9c26-0d99da922868 service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:51:40.503754 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:51:40.504418 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:51:40.504768 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:51:40.505047 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:51:40.506873 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:51:40.521477 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:51:40.522898 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:51:40.534538 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1462/2894] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:51:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:51:41.713362 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce43aaa-a010-43e8-8cbf-14dd947ee202 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:51:41.722714 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce43aaa-a010-43e8-8cbf-14dd947ee202 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:51:41.726406 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f677ed-1873-48f9-89cd-80b838cece31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b87f03cae6c4ca98a82dd5d981ed224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:51:41.727042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f677ed-1873-48f9-89cd-80b838cece31 service neutron] REQUEST_METHOD: `GET` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:51:41.727325 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f677ed-1873-48f9-89cd-80b838cece31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:51:41.727638 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f677ed-1873-48f9-89cd-80b838cece31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:51:41.729438 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-09f677ed-1873-48f9-89cd-80b838cece31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:51:41.743002 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f677ed-1873-48f9-89cd-80b838cece31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:51:41.744473 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f677ed-1873-48f9-89cd-80b838cece31 service neutron] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:51:41.756247 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1433/2895] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:51:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:51:50.529896 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=6 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:51:51.751327 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=6 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:51:53.305748 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:51:53.307238 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:51:53.374987 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61855d23-8aeb-4130-97de-4b1f0b5673fc service neutron] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:51:53.379918 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:51:53.380835 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:51:53.381116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:51:53.381384 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] PATH_INFO: `/v3/users/ca82a093a28f40f8a2bfc1f93df03493` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:51:53.382559 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca82a093a28f40f8a2bfc1f93df03493)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:51:53.384509 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:51:53.537950 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca82a093a28f40f8a2bfc1f93df03493'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:53.612332 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca82a093a28f40f8a2bfc1f93df03493'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:53.612987 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca82a093a28f40f8a2bfc1f93df03493'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:53.620258 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca82a093a28f40f8a2bfc1f93df03493'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:53.621003 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca82a093a28f40f8a2bfc1f93df03493'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:53.639051 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1463/2896] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:51:53 2022] DELETE /identity/v3/users/ca82a093a28f40f8a2bfc1f93df03493 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:51:53.882878 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f677ed-1873-48f9-89cd-80b838cece31 service neutron] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:51:53.893916 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f677ed-1873-48f9-89cd-80b838cece31 service neutron] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:51:53.952564 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:51:53.953346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:51:53.953627 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:51:53.953903 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] PATH_INFO: `/v3/projects/cf6354961eda4916a8324508b45a98d5` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:51:53.955113 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6354961eda4916a8324508b45a98d5)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:51:53.957077 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:51:54.077360 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6354961eda4916a8324508b45a98d5', 'request_id': 'req-3b39d823-c28f-4a5e-87aa-612f61fdca93'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.082913 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6354961eda4916a8324508b45a98d5', 'request_id': 'req-3b39d823-c28f-4a5e-87aa-612f61fdca93'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.089155 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6354961eda4916a8324508b45a98d5', 'request_id': 'req-3b39d823-c28f-4a5e-87aa-612f61fdca93'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.165692 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] The token cache is being invalidate because project cf6354961eda4916a8324508b45a98d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:51:54.165914 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.166545 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.170971 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1434/2897] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:51:53 2022] DELETE /identity/v3/projects/cf6354961eda4916a8324508b45a98d5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:51:54.172231 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:51:54.174871 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:51:54.188116 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec879-c6d1-4b67-a8f3-ca9a59775d43 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:51:54.243440 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:51:54.244337 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:51:54.244618 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:51:54.244894 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] PATH_INFO: `/v3/users/9b9ea01d4a4f456d88c6dbeeb7b541df` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:51:54.246071 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9ea01d4a4f456d88c6dbeeb7b541df)` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:51:54.248028 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:51:54.268747 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:51:54.448892 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9ea01d4a4f456d88c6dbeeb7b541df'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.526669 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9ea01d4a4f456d88c6dbeeb7b541df'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.527302 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9ea01d4a4f456d88c6dbeeb7b541df'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.535122 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9ea01d4a4f456d88c6dbeeb7b541df'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.544470 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.notifications [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9ea01d4a4f456d88c6dbeeb7b541df'} {{(pid=80345) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.553159 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1464/2898] 10.222.0.64 () {58 vars in 1223 bytes} [Mon Jun 27 09:51:54 2022] DELETE /identity/v3/users/9b9ea01d4a4f456d88c6dbeeb7b541df => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:51:54.802854 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] Authenticating user token {{(pid=80344) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:51:54.804349 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.fernet_utils [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80344) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:51:54.817933 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39d823-c28f-4a5e-87aa-612f61fdca93 admin admin] Validating token access rules against request {{(pid=80344) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:51:54.874919 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1fa6647d77b40cd8875b5b59ec1508d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd706bacf3a904a118b091c4d80341fee', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80344) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:51:54.875815 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:51:54.876097 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:51:54.876369 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] PATH_INFO: `/v3/projects/441a9b972f8b4bcdbdabf9162f1e979b` {{(pid=80344) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:51:54.878042 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=441a9b972f8b4bcdbdabf9162f1e979b)` {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:51:54.879998 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] RBAC: Authorization granted {{(pid=80344) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:51:54.959009 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:51:54.970140 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441a9b972f8b4bcdbdabf9162f1e979b', 'request_id': 'req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.975618 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441a9b972f8b4bcdbdabf9162f1e979b', 'request_id': 'req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:54.981582 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441a9b972f8b4bcdbdabf9162f1e979b', 'request_id': 'req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8'} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:55.039346 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] The token cache is being invalidate because project 441a9b972f8b4bcdbdabf9162f1e979b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80344) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 09:51:55.039671 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:55.040631 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG keystone.notifications [None req-6f12b9a8-6f1c-4706-8ede-057a2c5321b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80344) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 09:51:55.042495 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: [pid: 80344|app: 0|req: 1435/2899] 10.222.0.64 () {58 vars in 1232 bytes} [Mon Jun 27 09:51:54 2022] DELETE /identity/v3/projects/441a9b972f8b4bcdbdabf9162f1e979b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 09:52:04.549601 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:52:05.087213 dv-cinder-847730-1-iscsi devstack@keystone.service[80344]: DEBUG dbcounter [-] [80344] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80344) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 09:55:12.404398 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] Authenticating user token {{(pid=80345) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 09:55:12.405777 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:55:12.472755 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e910b1-6d4e-4111-9924-cbf241fa72f1 admin admin] Validating token access rules against request {{(pid=80345) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 09:55:12.531723 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c17c1-b75b-499c-8dbf-c79c1a124825 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d4d5fa82dc42e498fb22349921885e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84f2268a6c4c456293cdde00cf0026be', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80345) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 09:55:12.532373 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c17c1-b75b-499c-8dbf-c79c1a124825 service placement] REQUEST_METHOD: `GET` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 09:55:12.532658 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c17c1-b75b-499c-8dbf-c79c1a124825 service placement] SCRIPT_NAME: `/identity` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 09:55:12.532935 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c17c1-b75b-499c-8dbf-c79c1a124825 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80345) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 09:55:12.534727 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.fernet_utils [None req-fa5c17c1-b75b-499c-8dbf-c79c1a124825 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80345) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 09:55:12.547026 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c17c1-b75b-499c-8dbf-c79c1a124825 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 09:55:12.548694 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c17c1-b75b-499c-8dbf-c79c1a124825 service placement] RBAC: Authorization granted {{(pid=80345) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 09:55:12.559507 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: [pid: 80345|app: 0|req: 1465/2900] 10.222.0.64 () {60 vars in 1365 bytes} [Mon Jun 27 09:55:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 09:55:22.554866 dv-cinder-847730-1-iscsi devstack@keystone.service[80345]: DEBUG dbcounter [-] [80345] Writing DB stats keystone:SELECT=58 {{(pid=80345) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}